Error line contains null byte python

Python CSV error: line contains NULL byte, but no NULL byte found in the file

Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers ,Thanks for contributing an answer to Stack Overflow!,I’m getting this error when reading from a csv file: «Runtime Error! line contains NULL byte». Any idea about the root cause of this error?, Stack Overflow Public questions & answers

I’ve solved a similar problem with an easier solution:

import codecs csvReader = csv.reader(codecs.open('file.csv', 'rU', 'utf-16')) 

Answer by Dream Rios

If the csv module says that you have a «NULL» (silly message, should be «NUL») byte in your file, then you need to check out what is in your file. I would suggest that you do this even if using ‘rb’ makes the problem go away.,If you can see \x00 in the output (or \0 in your od -c output), then you definitely have NUL byte(s) in the file, and you will need to do something like this:,As @S.Lott says, you should be opening your files in ‘rb’ mode, not ‘rU’ mode. However that may NOT be causing your current problem. As far as I know, using ‘rU’ mode would mess you up if there are embedded \r in the data, but not cause any other dramas. I also note that you have several files (all opened with ‘rU’ ??) but only one causing a problem.,By the way, have you looked at the file (including the last few lines) with a text editor? Does it actually look like a reasonable CSV file like the other (no «NULL byte» exception) files?

I’m working with some CSV files, with the following code:

reader = csv.reader(open(filepath, "rU")) try: for row in reader: print 'Row read successfully!', row except csv.Error, e: sys.exit('file %s, line %d: %s' % (filename, reader.line_num, e)) 

And one file is throwing this error:

file my.csv, line 1: line contains NULL byte 

Following @JohnMachin’s comment below, I tried adding these lines to my script:

print repr(open(filepath, 'rb').read(200)) # dump 1st 200 bytes of file data = open(filepath, 'rb').read() print data.find('\x00') print data.count('\x00') 

And this is the output I got:

'\xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1\x00\x00\x00\x00\x00\x00\x00\x00\ . 8 13834 

Answer by Jianna Krueger

I’m trying to write a program that looks at a .CSV file (input.csv) and rewrites only the rows that begin with a certain element (corrected.csv), as listed in a text file (output.txt).,pandas.read_csv now handles the different UTF encoding when reading/writing and therefore can deal directly with null bytes,This is what my program looks like right now:,simply import csv module, write a dummy csv file, and then paste your data in that.

This is what my program looks like right now:

import csv lines = [] with open('output.txt','r') as f: for line in f.readlines(): lines.append(line[:-1]) with open('corrected.csv','w') as correct: writer = csv.writer(correct, dialect = 'excel') with open('input.csv', 'r') as mycsv: reader = csv.reader(mycsv) for row in reader: if row[0] not in lines: writer.writerow(row) 

Unfortunately, I keep getting this error, and I have no clue what it’s about.

Traceback (most recent call last): File "C:\Python32\Sample Program\csvParser.py", line 12, in for row in reader: _csv.Error: line contains NULL byte 

Answer by Mckinley Owens

I’m trying to write a program that looks at a .CSV file (input.csv) and rewrites only the rows that begin with a certain element (corrected.csv), as listed in a text file (output.txt).,The key was using the codecs module to open the file with the UTF-16 encoding, there are a lot more of encodings, check the documentation., Rakesh 2 Years ago I’ve solved a similar problem with an easier solution: import codecs csvReader = csv.reader(codecs.open(‘file.csv’, ‘rU’, ‘utf-16’)) The key was using the codecs module to open the file with the UTF-16 encoding, there are a lot more of encodings, check the documentation. ,Unfortunately, I keep getting this error, and I have no clue what it’s about.

This is what my program looks like right now:

import csv lines = [] with open('output.txt','r') as f: for line in f.readlines(): lines.append(line[:-1]) with open('corrected.csv','w') as correct: writer = csv.writer(correct, dialect = 'excel') with open('input.csv', 'r') as mycsv: reader = csv.reader(mycsv) for row in reader: if row[0] not in lines: writer.writerow(row) 

Unfortunately, I keep getting this error, and I have no clue what it’s about.

Traceback (most recent call last): File "C:\Python32\Sample Program\csvParser.py", line 12, in for row in reader: _csv.Error: line contains NULL byte 

Answer by Briar Whitney

When trying to import a csv and put it in a table I get this error.,File “event:actionPerformed”, line 62, in ,Traceback (most recent call last):,I just want to grab information from a CSV and write specific information to certain tags. Maybe there is a easier way to do this.

#Master Import
file = open(“locationoffile\ITEM_MASTER2.csv”)
csvData = csv.reader(file)

header = csvData.next() pyData = [] for row in csvData: pyData.append(row) finalData = system.dataset.toDataSet(header,pyData) event.source.parent.getComponent('Master Table').data = finalData 

Answer by Olive Bridges

Make an error file like this, print it out in PyCharm, and then copy the printed result to a CSV file,A line of content read by CSV contains a null byte which in Python is \x00, Solution to the problem of failure to elect leaders when offline service is reported in Nacos ,Why this terrible mistake? 1. The data in it has this thing 2. The file is an excel file, but the result is saved as CSV 3. Multiple processes read and write to the same file

A line of content read by CSV contains a null byte
which in Python is \x00

In [1]: a = '\0' In [2]: a Out[2]: '\x00' In [3]: b = '\x00' In [4]: b Out[4]: '\x00' In [5]: a == b Out[5]: True 

Make an error file like this, print it out in PyCharm, and then copy the printed result to a CSV file

with open('file.csv', 'r') as f: reader = csv.reader(f) next(reader) 

Error: line contains NULL byte
. So, how do you do that
Modify the code

with open('file.csv', 'r') as f: reader = csv.reader(_.replace('\x00', '') for _ in f) next(reader) 

Answer by Beatrice Castillo

#you may have null byte in csv , you can remove on linux with this command sed 's/\x0/ /g' original_file.csv > fixed_file.csv

Answer by Analia Wang

I wrote the following code to transform the data in a csv file and write to another csv file:,The code worked half way through the file and stopped due to an error raised, that said ‘for row in csv.reader, line contains NULL byte’. I want to skip this error and proceed. Despite the except statement, it still stopped. Does anyone know how to solve this issue?,import csvimport refp = r’C:\data\input.csv’fpw = r’C:\data\output.csv’with open(fp, ‘rb’) as input, open(fpw, ‘wb’) as output: for row in csv.reader(input:(, stop = row[11] # find a particular field extr = re.search(r»\[([A-Za-z0-9_]+)\]», stop) # extract value enclosed by brackets in that field stop_id = str(extr.group(1)) row[11] = stop_id # replace the original field with the extracted value repl_row = ‘,’.join(row) + ‘\n’ output.write(repl_row) # write the tweaked row to another csv file except csv.Error: pass

import csv
import re
fp = r ‘C:\data\input.csv’
fpw = r ‘C:\data\output.csv’

with open ( fp , ‘rb’ ) as input , open ( fpw , ‘wb’ ) as output :
for row in csv . reader ( input 🙁

import csvimport refp = r'C:\data\input.csv'fpw = r'C:\data\output.csv'with open(fp, 'rb') as input, open(fpw, 'wb') as output: for row in csv.reader(input:(

Answer by Mason Leal

I think this has been asked before, but it has been awhile and I think needs to be re-addressed. Why doesn't the CSV module handle NULL bytes? Let me do some examples, ascii = [chr(n) for n in range(256)] #No errors print(ascii) #No errors print(dict(zip(ascii,ascii))) #No errors open('temp','r').writelines(ascii) #No errors f=open('temp','r') for line in f: print(line) #No errors f.close() Python hsndles every ascii chr, DEL, ESC, RETURN, etc. It displays those characters, uses them as keys or values, etc. But now try, import csv data = csv.DictReader(ascii) for line in data: print(line) #NULL Byte Error But we can quick fix this easily, ascii.pop(0) data = csv.DictReader(ascii) for line in data: print(line) #No Errors Doesn't it seem odd that we allow the use of every chr as keys, vslues, prints, etc. but then we hold a special exception for using the csv module and yhat special exception is not the ESC or DEL or any other non-printable chr, the exception is for the NULL?

Источник

How to fix python csv error: line contains null byte?

When working with CSV files in Python, sometimes you may encounter the error «line contains NULL byte». This error occurs when the CSV file contains a null byte in one of the rows, which is an illegal character in a string. The presence of a null byte in the string can cause issues when reading or processing the data in the CSV file. In this article, we will discuss a few methods to fix this error and successfully read the CSV file in Python.

Method 1: Filter Out NULL Bytes

To fix the Python CSV error «line contains NULL byte», you can use the «Filter Out NULL Bytes» method. Here are the steps to do it with code examples:

import csv with open('file.csv', 'rb') as csv_file: csv_reader = csv.reader(csv_file) # rest of the code

import csv with open('file.csv', 'rb') as csv_file: csv_reader = csv.reader(filter(lambda row: '\0' not in row, csv_file)) # rest of the code

import csv with open('file.csv', 'rb') as csv_file: csv_reader = csv.reader(filter(lambda row: '\0' not in row, csv_file)) for row in csv_reader: # process the row as needed

Here is the complete code example:

import csv with open('file.csv', 'rb') as csv_file: csv_reader = csv.reader(filter(lambda row: '\0' not in row, csv_file)) for row in csv_reader: # process the row as needed

In summary, the «Filter Out NULL Bytes» method involves opening the CSV file in binary mode, using the filter() function to remove the NULL bytes from each row, and then iterating through each row and processing it as needed.

Method 2: Use the Error Handling Feature in the CSV Library

To fix the «Python CSV error: line contains NULL byte», you can use the error handling feature in the CSV library. Here are the steps to do it:

1. Open the CSV file with the csv module.
2. Set the csv.Error exception to catch the error.
3. Use the replace method to remove the NULL byte from the line.
4. Continue reading the CSV file.

Here is an example code snippet:

import csv with open('file.csv', 'r') as csv_file: csv_reader = csv.reader(csv_file) try: for row in csv_reader: # process the row pass except csv.Error as e: # remove the NULL byte and continue reading csv_reader = csv.reader(x.replace('\0', '') for x in csv_file) for row in csv_reader: # process the row pass

In this example, we open the CSV file and create a csv.reader object. We then use a try and except block to catch the csv.Error exception. If the exception is caught, we remove the NULL byte from the line using the replace method and create a new csv.reader object to continue reading the file.

Note that this method may not work for all cases of the «Python CSV error: line contains NULL byte».

Method 3: Read the CSV File in Binary Mode and Decode

To fix the «Python CSV error: line contains NULL byte» issue, you can try reading the CSV file in binary mode and decode it. Here are the steps to do it:

1. Open the CSV file in binary mode using the built-in open() function.
2. Read the contents of the file using the read() method.
3. Decode the file contents using the decode() method with the appropriate encoding (e.g. UTF-8).
4. Pass the decoded contents to the csv.reader() function to parse the CSV data.

Here’s an example code snippet that demonstrates this approach:

import csv with open('file.csv', 'rb') as file: contents = file.read().decode('utf-8') reader = csv.reader(contents.splitlines()) for row in reader: print(row)

In this example, we open the CSV file in binary mode using the ‘rb’ mode. We then read the contents of the file into a variable called contents using the read() method. Next, we decode the contents using the decode() method with the UTF-8 encoding.

Finally, we pass the decoded contents to the csv.reader() function to parse the CSV data. The splitlines() method is used to split the contents into separate lines before passing it to the csv.reader() function.

This approach should help you fix the «Python CSV error: line contains NULL byte» issue when working with CSV files.

Источник