Python - Unicode & amp; double backslashes

advertisements

I scrapped a webpage with BeautifulSoup. I got great output except parts of the list look like this after getting the text:

list = [u'that\\u2019s', u'it\\u2019ll', u'It\\u2019s', u'don\\u2019t', u'That\\u2019s', u'we\\u2019re', u'\\u2013']

My question now is how to get rid or replace these double backslashes with the special characters they are.

If i print the first the first element of the example list the output looks like

print list[0]
that\u2019s

I already read a lot of other questions / threads about this topic but I ended up being even more confused, as I am a beginner considering unicode / encoding / decoding.

I hope that someone could help me with this issue.

Thanks! MG


the problem here is that the site ended up double encoding those unicode arguments, just do the following:

ls = [u'that\\u2019s', u'it\\u2019ll', u'It\\u2019s', u'don\\u2019t', u'That\\u2019s', u'we\\u2019re', u'\\u2013']

ls = map(lambda x: x.decode('unicode-escape'), ls)

now you have a list with properly unicode encoded strings:

for a in ls:
   print a