• Register
Welcome to Kodlogs, programming questions and answer website.
0 votes
pandas to csv: I am dealing with dataframe in pandas which I would like to write to a CSV file, how to achieve this?
by (1.4k points)  
reshown by

1 Answer

0 votes

How we can save or write these videos so let's say we've loaded our data frame we've manipulated the data maybe updated it changed a few values here and there hired new columns remove rows things like that and now we wanted to save it may be to send it somebody else so to you know save it for later processing it's quite easy to do that with pandas all we need to do is to use the to CSV function as we're gonna see so let's import our did whole pandas as PDS as usual we've seen this raw data dictionary before I'll forward it from one of the previous videos from one of the previous Jupiter notebook the notebooks that we've created so let's create a dictionary and then create a data frame out of it and now we can save that data frame as a CSV and the function to do is this is a function inside the data frame now it's not coming from pandas directly it's not like PD dot reaches V as we saw before but now we have the data frame and we use a function inside the data frame module to do the saving if I run that it'll save it to slash home slash new audience like download slash data dot CSV do that and if I display the contents the data frame contents the one we've said it looks like this a few interesting things to discuss here the first one is by default the separator is a comma and we can by the way have a look at the parameters that it uses and is quite you know it's quite actually a few of them so the separators comma by default the n/a or presentation they're not available what you want to do with that what I want the header or not by default it's true this index equals index equals true is quite interesting we'll see what that means what else do we have the encoding is none by default you can actually use compression if you want and that's quite interesting to reduce the size the quote char I've not spoken about this this is the issue when you have you know if you are for example you your separators comma and then one of your values has comma within it by default if we don't enclose that into quotes then it'll be that comma will be read as a separator and it can lead to errors in reading or writing the files so we can surround that or enclose and close that into double quotes the line Terminator by default is /n and so on and so forth.

Several options so let's have a look at the stuff that we've just saved and if we read it so we've saved it to this file here if we read it if we read it DF we've seen this before PD lot read CSV save it and data frame in the variable called data frame 1 and if we display it then we notice we have an extra column here unnamed 0 and then a value the values are 0 1 2 3 4 & 5 like the index the reason for this is that as we saw here the index is true by default where is it yeah index and pandas by default saves sorry pandas by default save the index as a new column and we can disable that by saying index equals false if we run that you have a look at the data frame we read it again and display it now and you notice that column has gone I didn't have a column name or label partners why default gave it unnamed here but like we said before it's quite convenient sometimes to disable the index what else we can also change the separating character instead of comma we can say for example let's make it /t save and then when we read we're gonna have to if we want to specify that then it'll be like a mess but we need to say separated equals / the way and then our data frame will read nicely right what else can we talk about here I think I think that's that's good enough for this video and if anything comes up you know in the future we'll deal with it but generally this is how you read and write is V Files and this is how you change the separator turn the index into false and we spoke about using the first row as the header yes or no disable I'm sorry true or false disable or enable and always have a look at the documentation by doing shift-tab here on your jupiter notebook you can you know get it larger to more display space and you can read through the variables i'm sorry - through the parameters that you would like to use this so the compression for example is quite interesting you can say compression equals gzip or XZ or be easy-peasy - subdued wanted to reduce this file so as you can play about with a quote a quote char and so on and so forth . 

by (9.7k points)