Uni-WHAT!?



Unicode errors in python 2.7


Ever seen this at the top of a python file?

 import sys
reload(sys)
sys.setdefaultencoding('utf8')

 If you see this it means the person who wrote the python code received a unicode error, didn't know how to fix it properly, then Google'd the error and found this quick solution. Here's a good stack overflow post on why not to use setdefaultencoding()


 The short answer is, in Python 2 the default encoding is ASCII. When it encounters other encodings it will raise errors. When you change the default encoding you can't be sure whats going out of your code is what came in. Also, when you reload a module, you actually get two copies of the module in your runtime which can cause some weird issues.

 So how to deal with different encodings? use decode() and deal with the text as it comes in. Use var.decode(encoding) to decode a byte string as text. Then use var.encode(encoding) to encode a text string as bytes.

Better yet, Python3 treats all strings a sequences of Unicode characters. There is no such thing as a Python string encoded in utf-8. So maybe just use python 3 if you can!

Comments

Post a Comment

Popular posts from this blog

Upcasting in python

Upcasting Link to official Docs Have you ever run into a scenario where you have set your column type to int but when you go to display it either in a report or visualization it comes out a float? This happens because of something called upcasting. "Types can potentially be upcasted when combined with other types, meaning they are promoted from the current type (e.g. int to float)." Lets start with a DataFrame of 1 column and 8 rows where the values are random numbers from the normal distribution. >>> import pandas as pd >>>   import numpy as np >>>   df1 = pd.DataFrame(np.random.randn(8, 1), columns=['A'], dtype='float32') >>>   df1    A 0  0.406792 1  0.810450 2  1.161985 3 -1.402411 4  1.385434 5 -1.091746 6  0.018586 7 -0.606741 Now lets create a 3x8 DataFrame >>>   df2 = pd.DataFrame( dict(                  A =...
Read more

Chart Types & Styles

Image
Chart Types & Styles What is your process when you set out to make a data vizualization? Do you sketch what you are trying to show or discover? If using Tableau do you just start throwing variables into frame? Below is a useful though starter as you plan your data viz. Chart suggestions a thought starter Visualization of most common business data only requires 2 dimensional representation. Adding a third variable can be confusing for people who don't work with data everyday. So lets keep it simple and go through some of the decision making steps when deciding which visualization to use.  Three important things to consider are: Number of variables If you have more than four you may be in for a cluttered, unclear chart. Type of variables Numeric- Discrete or continuous Categorial - related or unrelated categories Association you are trying to show Relationship Multiple variables. Dependent or Independent. Comparison Difference or Similarity. Trends over...
Read more