Python文件编码---gbk?OR utf8?

﹏ヽ暗。殇╰゛Y 2022-03-19 14:46 606阅读 0赞

## [Python文件编码---gbk?OR utf8?][Python_---gbk_OR utf8] ##

windows文件名的编码是cp936的，你在使用中文文件名的时候转下码就行了。

比如你python文件编码是utf8

\# -\*- coding: utf-8 -\*-

he='开心.mp3'  
f=open(he.decode('utf-8').encode('cp936'),'w')  
f.close()

\-----------------------------------------------  
如果是GB2312,则是最佳选择.GB2312含7千多字.GB2312是国内外软件普遍接受和支持的8bit双字节中文编码.

但你是gbk.GBK是扩展的GB2312,大部分软件不支持它.用GBK时,论坛内容显示时,一些非GB2312中文字会显示成空白方块.

utf8好,UTF8是unicode的传送型式.主流浏览器IE和netscape都支持.

\[看一下IE的View->Encoding下的可接受编码,看一下netscape的View->Charactercoding下的可接受编码,找得到GB2312和UTF8,找不到GBK!这就是在GBK和UTF8两种编码中只好选UTF8的原因.\]

\------------------------------------------------  
用编码gbk还是utf8  
\------------------------------------------------  
\# -\*- coding: UTF-8 -\*- 这是个注释吗？  
这是用来说明你的Python源程序文件用使用的编码。缺省情况下你的程序需要使用ascii码来写，但如果在其中写中文的话，python解释器一般会报错，但如果加上你所用的文件编码，python就会自动处理不再报错。

上述格式还可以写成：

\#coding=utf-8  
或  
\#coding:utf-8  
\-------------------------------------------------  
Defining Python Source Code Encodings

Abstract  
This PEPproposes to introduce a syntax to declare the encoding of  
a Pythonsource file. The encoding information is then used by the  
Pythonparser to interpret the file using the given encoding. Most  
notably thisenhances the interpretation of Unicode literals in  
the sourcecode and makes it possible to write Unicode literals  
using e.g.UTF-8 directly in an Unicode aware editor.

Problem  
In Python2.1, Unicode literals can only be written using the  
Latin-1based encoding "unicode-escape". This makes the  
programmingenvironment rather unfriendly to Python users who live  
and work innon-Latin-1 locales such as many of the Asian  
countries.Programmers can write their 8-bit strings using the  
favoriteencoding, but are bound to the "unicode-escape" encoding  
for Unicodeliterals.

Proposed Solution  
I propose tomake the Python source code encoding both visible and  
changeableon a per-source file basis by using a special comment  
at the topof the file to declare the encoding.

To makePython aware of this encoding declaration a number of  
conceptchanges are necessary with respect to the handling of  
Pythonsource code data.

Defining the Encoding  
Python willdefault to ASCII as standard encoding if no other  
encodinghints are given.

To definea source code encoding, a magic comment must  
be placedinto the source files either as first or second  
line in thefile, such as:

\# coding=<encoding name>

or (usingformats recognized by popular editors)

\#!/usr/bin/python  
\# -\*- coding: <encoding name> -\*-

\#!/usr/bin/python  
\# vim: set fileencoding=<encodingname> :

Moreprecisely, the first or second line must match the regular  
expression"coding\[:=\]\\s\*(\[-\\w.\]+)". The first group of this  
expressionis then interpreted as encoding name. If the encoding  
is unknownto Python, an error is raised during compilation. There  
must not beany Python statement on the line that contains the  
encodingdeclaration.

To aidwith platforms such as Windows, which add Unicode BOM marks  
to thebeginning of Unicode files, the UTF-8 signature  
'\\xef\\xbb\\xbf' will be interpreted as 'utf-8' encoding aswell  
(even if nomagic encoding comment is given).

If asource file uses both the UTF-8 BOM mark signature and a  
magicencoding comment, the only allowed encoding for the comment  
is'utf-8'.  Any other encoding will cause anerror.

Examples  
These aresome examples to clarify the different styles for  
defining thesource code encoding at the top of a Python source  
file:

1. Withinterpreter binary and using Emacs style file encoding  
comment:

\#!/usr/bin/python  
\# -\*- coding: latin-1 -\*-  
import os, sys  
...

\#!/usr/bin/python  
\# -\*- coding: iso-8859-15 -\*-  
import os, sys  
...

\#!/usr/bin/python  
\# -\*- coding: ascii -\*-  
import os, sys  
...

2.Without interpreter line, using plain text:

\# This Python file uses the following encoding: utf-8  
import os, sys  
...

3. Texteditors might have different ways of defining the file's  
encoding, e.g.

\#!/usr/local/bin/python  
\# coding: latin-1  
import os, sys  
...

4.Without encoding comment, Python's parser will assume ASCII  
text:

\#!/usr/local/bin/python  
import os, sys  
...

5.Encoding comments which don't work:

Missing "coding:" prefix:

\#!/usr/local/bin/python  
\# latin-1  
import os, sys  
...

Encoding comment not on line 1 or 2:

\#!/usr/local/bin/python  
\#  
\# -\*- coding: latin-1 -\*-  
import os, sys  
...

Unsupported encoding:

\#!/usr/local/bin/python  
\# -\*- coding: utf-42 -\*-  
import os, sys  
...

Concepts  
The PEP isbased on the following concepts which would have to be  
implementedto enable usage of such a magic comment:

1. Thecomplete Python source file should use a single encoding.  
Embedding of differently encoded data is not allowed and will  
result in a decoding error during compilation of the Python  
source code.

Any encoding which allows processing the first two lines inthe  
way indicated above is allowed as source code encoding, this  
includes ASCII compatible encodings as well as certain  
multi-byte encodings such as Shift\_JIS. It does not include  
encodings which use two or more bytes for all characters like  
e.g. UTF-16. The reason for this is to keep the encoding  
detection algorithm in the tokenizer simple.

2.Handling of escape sequences should continue to work as itdoes  
now, but with all possible source code encodings, that is  
standard string literals (both 8-bit and Unicode) are subjectto  
escape sequence expansion while raw string literals onlyexpand  
a very small subset of escape sequences.

3.Python's tokenizer/compiler combo will need to be updated to  
work as follows:

1. read the file

2. decode it into Unicode assuming a fixed per-file encoding

3. convert it into a UTF-8 byte string

4. tokenize the UTF-8 content

5. compile it, creating Unicode objects from the given Unicodedata  
and creating string objects from the Unicode literal data  
by first reencoding the UTF-8 data into 8-bit string data  
using the given file encoding

Note that Python identifiers are restricted to the ASCII  
subset of the encoding, and thus need no further conversion  
after step 4.

Implementation  
Forbackwards-compatibility with existing code which currently  
usesnon-ASCII in string literals without declaring an encoding,  
theimplementation will be introduced in two phases:

1. Allownon-ASCII in string literals and comments, by internally  
treating a missing encoding declaration as a declaration of  
"iso-8859-1". This will cause arbitrary byte strings to  
correctly round-trip between step 2 and step 5 of the  
processing, and provide compatibility with Python 2.2 for  
Unicode literals that contain non-ASCII bytes.

A warning will be issued if non-ASCII bytes are found in the  
input, once per improperly encoded input file.

2. Removethe warning, and change the default encoding to "ascii".

Thebuiltin compile() API will be enhanced to accept Unicode as  
input. 8-bitstring input is subject to the standard procedure for  
encodingdetection as described above.

If aUnicode string with a coding declaration is passed tocompile(),  
aSyntaxError will be raised.

SUZUKIHisao is working on a patch; see \[2\] for details. A patch  
implementingonly phase 1 is available at \[1\].

Phases  
Implementation of steps 1 and 2 above were completed in 2.3,  
except forchanging the default encoding to "ascii".

Thedefault encoding was set to "ascii" in version 2.5.  
  
Scope  
This PEPintends to provide an upgrade path from the current  
(more-or-less) undefined source code encoding situation to amore  
robust andportable definition.

References  
\[1\] Phase 1implementation:  
[http://python.org/sf/526840][http_python.org_sf_526840]  
\[2\] Phase 2implementation:  
[http://python.org/sf/534304][http_python.org_sf_534304]

History  
1.10 andabove: see CVS history  
1.8: Added'.' to the coding RE.  
1.7: Addedwarnings to phase 1 implementation. Replaced the  
Latin-1 default encoding with the interpreter's default  
encoding. Added tweaks to compile().  
1.4 - 1.6:Minor tweaks  
1.3: Workedin comments by Martin v. Loewis:  
UTF-8 BOM mark detection, Emacs style magic comment,  
two phase approach to the implementation

Thisdocument has been placed in the public domain.

### [【mysql + python】UnicodeEncodeError: 'latin-1' codec can't encode characters in position 28-29: ordinal not in range(256)][mysql _ python_UnicodeEncodeError_ _latin-1_ codec can_t encode characters in position 28-29_ ordinal not in range_256]    ###

数据库表的字符集是utf-8，如果插入的数据是汉字，那么就需要做字符集转换。python中的方法是黄色底纹的代码

def getinfo():

connection = MySQLdb.connect(host='10.20.149.247',user='user', passwd='bkeep', db='ep')

cursor = connection.cursor()

sqlip = """ select distinct names\_ip from xingyi\_dns where types='A' """

sql\_update = """ update xingyi\_dns set idc='%s' where names\_ip = '%s' and types = 'A' """

cursor.execute(sqlip)

count = cursor.fetchall()

\#print type(count)

for i in count:

ip = i\[0\]

idc\_a = get\_idc\_ip(ip)

if isinstance(idc\_a, unicode):

idc\_a = idc\_a.encode('utf-8')

cursor.execute(sql\_update %(idc\_a,ip))

print ip

getinfo()

再执行就成功了。

[Python_---gbk_OR utf8]: http://blog.sina.com.cn/s/blog_6c3919650100zfar.html
[http_python.org_sf_526840]: http://python.org/sf/526840
[http_python.org_sf_534304]: http://python.org/sf/534304
[mysql _ python_UnicodeEncodeError_ _latin-1_ codec can_t encode characters in position 28-29_ ordinal not in range_256]: http://bkeep.blog.163.com/blog/static/123414290201212925223441/