• Register
0 votes
693 views

Problem :

I wanted to read all the csv files from my folder and after that concatenate them to create a big csv file. The structure of all my files was the same, After concatenating I want to save it and read it again. All this was done by me using Pandas. But I am facing the Error while reading.

Below is my csv reader code :-

import csv
with open("C:\mywork\DATA\Raw_data\\store.csv",'rb') as csv:
csvreader = csv. csvreader (csv)
l = list(csvreader)

My Error:-

Error                                     Traceback (most recent call last)
<ipython-input-36-9249469f31a6> in <module>()
1 with open('C:\ mywork \DATA\Raw_data\\store.csv', 'rb') as csv:
2     csvreader = csv. csvreader (csv)
----> 3     l = list(csvreader)
Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
6 5 3
7,540 points

Please log in or register to answer this question.

1 Answer

0 votes

Solution :

I also faced this error in recent past, the root cause I found was that there were some carriage returns "\r" in my data that pandas was using as the line terminator similar to "\n". I guess I want to share it here as that may be a common reason behind this error

The solution which I found was to add the lineterminator='\n' into my read_csv function as follows:

df_clean = pd.read_csv('test_myerror.csv',lineterminator='\n')

 

9 7 4
38,600 points

Related questions

0 votes
1 answer 4.4K views
4.4K views
Problem : I have the pandas data frame with some of the categorical predictors or variables as 0 & 1, and some of the numeric variables. When I fit that to a stasmodel like below : est = sm.OLS(y, X).fit() It throws the below error : Pandas data ... hundreds of variables. For that I have concatenated the 3 pandas DataFrames to come up with the final DataFrame to be used in the model building.
asked Dec 18, 2019 alecxe 7.5k points
0 votes
1 answer 6 views
6 views
Problem: I am a beginner to python and am at the moment having trouble using the command line. I have a script test.py (which only contains print("Hello.")), and it is located in the map C:\Python27. In my system variables, I have specified python to be C:\Python27 (I ... test.py in the command line, but when I do so I get this: File "<stdin>", line 1 python test.py ^ SyntaxError: invalid syntax
asked 6 days ago Mashhoodch 5.6k points
0 votes
1 answer 2K views
2K views
Problem : MySample.csv contains the below details : NAME Id No Dept Tommy 1 12 CS Jimmy 2 35 EC Bonny 3 21 IT Franky 4 61 EE And my Python file contains the below code : import csv myifile = open('mysample.csv', "rb") read = csv.reader(myifile) for row in read : ... in for row in read : _csv.Error: iterator should return strings, not bytes (did you open the file in text mode?) How should I fix it?
asked Jan 2, 2020 alecxe 7.5k points
0 votes
1 answer 493 views
493 views
Probem : I am getting bellow error new-line character seen in unquoted field - do you need to open the file in universal-newline mode?
asked Nov 7, 2019 peterlaw 6.9k points
0 votes
2 answers 1.1K views
1.1K views
Problem : I am new to RandomForest model. While predicting my test data using the RandomForest model I am often facing below ValueError. &ldquo;Input contains nan, infinity or a value too large for dtype('float64')&rdquo; I have spent more than two days on the above error but I am unable to fix above error. Can somebody help me in fixing above error?
asked Feb 24, 2020 mphil 2.3k points
0 votes
1 answer 21 views
21 views
Problem: I need a quick PHP code that will export my records to a csv file. The fields in the record includes the item_id, item_name, item_description, item_price, item_qty. I needed these headers included in the csv.
asked Oct 31, 2020 RJ Lam 650 points
0 votes
1 answer 8 views
8 views
Problem: I wrote some code where I find common key-value pairs between two dictionaries as follows: d_inter = dict(set(message.iteritems()).intersection(v.iteritems())) This works fine, but when messagethere keyis a type in dictionaries list, I get an error TypeError: ... when we try to use listas keyin any dictionary, but I am not doing anything like this here. Please help me fix this.
asked Dec 24, 2020 sasha 5.3k points
0 votes
1 answer 1.1K views
1.1K views
Problem : When I run my Python code I get the following exception: _csv.error: iterator should return strings, not bytes (did you open the file in text mode?)
asked Nov 18, 2019 peterlaw 6.9k points
0 votes
1 answer 11 views
11 views
Problem: I want to read csv file by using c++ so here is my code int main(){ ifstream classFile("class.csv"); vector<string> classData; while (getline(classFile, line,',')) // there is input overload classfile { classData.push_back(line); } } here is my question ... +/ Library / John is there anyway that I can separate my last column from first of next row? Thank you and sorry that it is confusing
asked Dec 26, 2020 Mashhoodch 5.6k points
0 votes
1 answer 5 views
5 views
Problem: The CSV file contains 37 attributes + the label (last column). it doesn't contain header or index. Exp of the 1st row: 63651000000.0,63651000000.0,153.1,0,0,0,0,0,0,5,1,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1 ... "csv2libsvm.py", line 47, in <module> headers = reader.next() AttributeError: '_csv.reader' object has no attribute 'next' Do you have any idea about the problem ?
asked 6 days ago Mashhoodch 5.6k points