dump of lists into csv export
-2
I need to add column headers and then export the following to csv.
But when I export the list, I get the last line with each of the characters in columns.
This is the output:
If I implement df=pd.DataFrame([d])
I get the following :
Code that creates this list is as follows
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d= parts[2:4]
df=pd.DataFrame(d)
print(df)
python pandas
edited Nov 12 at 0:23
eyllanesc
73.4k103056
asked Nov 11 at 23:29
tabarneck
226
add a comment |
-2
I need to add column headers and then export the following to csv.
But when I export the list, I get the last line with each of the characters in columns.
This is the output:
If I implement df=pd.DataFrame([d])
I get the following :
Code that creates this list is as follows
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d= parts[2:4]
df=pd.DataFrame(d)
print(df)
python pandas
edited Nov 12 at 0:23
eyllanesc
73.4k103056
asked Nov 11 at 23:29
tabarneck
226
2
Please replace the screenshots with the actual data. Screenshots of data, code, and errors do not belong on SO.
– DYZ
Nov 11 at 23:35
@DYZ but I am unable to get the data to csv hence why I am asking the question. I cannot copy paste the results for some reason from the command prompt. Its a lot of data by the way and its just a small fraction of it.
– tabarneck
Nov 11 at 23:47
What type of object isd? Why are you passing it topd.DataFrameas a list (e.g. in). Have you triedpd.DataFrame(d)?
– dmitriys
Nov 11 at 23:50
I get something like this if I do without the alon-De-Coiffure-Pour-Hommes-Laurier 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 Kings-Barbershop 0 0 Longueuil 1 Kings-Barbershop
– tabarneck
Nov 11 at 23:52
1
I will add the code to the post
– tabarneck
Nov 11 at 23:54
add a comment |
-2
-2
-2
I need to add column headers and then export the following to csv.
But when I export the list, I get the last line with each of the characters in columns.
This is the output:
If I implement df=pd.DataFrame([d])
I get the following :
Code that creates this list is as follows
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d= parts[2:4]
df=pd.DataFrame(d)
print(df)
python pandas
edited Nov 12 at 0:23
eyllanesc
73.4k103056
asked Nov 11 at 23:29
tabarneck
226
I need to add column headers and then export the following to csv.
But when I export the list, I get the last line with each of the characters in columns.
This is the output:
If I implement df=pd.DataFrame([d])
I get the following :
Code that creates this list is as follows
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d= parts[2:4]
df=pd.DataFrame(d)
print(df)
python pandas
python pandas
edited Nov 12 at 0:23
eyllanesc
73.4k103056
asked Nov 11 at 23:29
tabarneck
226
edited Nov 12 at 0:23
eyllanesc
73.4k103056
asked Nov 11 at 23:29
tabarneck
226
edited Nov 12 at 0:23
eyllanesc
73.4k103056
edited Nov 12 at 0:23
eyllanesc
73.4k103056
edited Nov 12 at 0:23
eyllanesc
73.4k103056
73.4k103056
asked Nov 11 at 23:29
tabarneck
226
asked Nov 11 at 23:29
tabarneck
226
asked Nov 11 at 23:29
tabarneck
226
226
2
Please replace the screenshots with the actual data. Screenshots of data, code, and errors do not belong on SO.
– DYZ
Nov 11 at 23:35
@DYZ but I am unable to get the data to csv hence why I am asking the question. I cannot copy paste the results for some reason from the command prompt. Its a lot of data by the way and its just a small fraction of it.
– tabarneck
Nov 11 at 23:47
What type of object isd? Why are you passing it topd.DataFrameas a list (e.g. in). Have you triedpd.DataFrame(d)?
– dmitriys
Nov 11 at 23:50
I get something like this if I do without the alon-De-Coiffure-Pour-Hommes-Laurier 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 Kings-Barbershop 0 0 Longueuil 1 Kings-Barbershop
– tabarneck
Nov 11 at 23:52
1
I will add the code to the post
– tabarneck
Nov 11 at 23:54
add a comment |
2
Please replace the screenshots with the actual data. Screenshots of data, code, and errors do not belong on SO.
– DYZ
Nov 11 at 23:35
@DYZ but I am unable to get the data to csv hence why I am asking the question. I cannot copy paste the results for some reason from the command prompt. Its a lot of data by the way and its just a small fraction of it.
– tabarneck
Nov 11 at 23:47
What type of object isd? Why are you passing it topd.DataFrameas a list (e.g. in). Have you triedpd.DataFrame(d)?
– dmitriys
Nov 11 at 23:50
I get something like this if I do without the alon-De-Coiffure-Pour-Hommes-Laurier 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 Kings-Barbershop 0 0 Longueuil 1 Kings-Barbershop
– tabarneck
Nov 11 at 23:52
1
I will add the code to the post
– tabarneck
Nov 11 at 23:54
2
2
Please replace the screenshots with the actual data. Screenshots of data, code, and errors do not belong on SO.
– DYZ
Nov 11 at 23:35
Please replace the screenshots with the actual data. Screenshots of data, code, and errors do not belong on SO.
– DYZ
Nov 11 at 23:35
@DYZ but I am unable to get the data to csv hence why I am asking the question. I cannot copy paste the results for some reason from the command prompt. Its a lot of data by the way and its just a small fraction of it.
– tabarneck
Nov 11 at 23:47
@DYZ but I am unable to get the data to csv hence why I am asking the question. I cannot copy paste the results for some reason from the command prompt. Its a lot of data by the way and its just a small fraction of it.
– tabarneck
Nov 11 at 23:47
What type of object is
d? Why are you passing it to pd.DataFrame as a list (e.g. in pd.DataFrame(d)?– dmitriys
Nov 11 at 23:50
What type of object is
d? Why are you passing it to pd.DataFrame as a list (e.g. in pd.DataFrame(d)?– dmitriys
Nov 11 at 23:50
I get something like this if I do without the alon-De-Coiffure-Pour-Hommes-Laurier 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 Kings-Barbershop 0 0 Longueuil 1 Kings-Barbershop
– tabarneck
Nov 11 at 23:52
I get something like this if I do without the alon-De-Coiffure-Pour-Hommes-Laurier 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 Kings-Barbershop 0 0 Longueuil 1 Kings-Barbershop
– tabarneck
Nov 11 at 23:52
1
1
I will add the code to the post
– tabarneck
Nov 11 at 23:54
I will add the code to the post
– tabarneck
Nov 11 at 23:54
add a comment |
1 Answer
1
active
oldest
votes
1
I think the issue is that for each iteration of the loop you're creating a new DataFrame; whereas what I imagine you probably want is a single DataFrame with all of the values for d. Try the below:
# Instantiate an empty list to collect values of `d` for each iteration
d_values =
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d = parts[2:4]
# Append `d` to your list, creating a list of lists
d_values.append(d)
# Once your loop is done, create a DataFrame from your list of lists
df = pd.DataFrame(d_values, columns=['col1name','col2name'])
answered Nov 12 at 1:30
dmitriys
15119
Thank you so much! really appreciate your help!
– tabarneck
Nov 12 at 4:08
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53254297%2fdump-of-lists-into-csv-export%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
I think the issue is that for each iteration of the loop you're creating a new DataFrame; whereas what I imagine you probably want is a single DataFrame with all of the values for d. Try the below:
# Instantiate an empty list to collect values of `d` for each iteration
d_values =
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d = parts[2:4]
# Append `d` to your list, creating a list of lists
d_values.append(d)
# Once your loop is done, create a DataFrame from your list of lists
df = pd.DataFrame(d_values, columns=['col1name','col2name'])
answered Nov 12 at 1:30
dmitriys
15119
Thank you so much! really appreciate your help!
– tabarneck
Nov 12 at 4:08
add a comment |
1
I think the issue is that for each iteration of the loop you're creating a new DataFrame; whereas what I imagine you probably want is a single DataFrame with all of the values for d. Try the below:
# Instantiate an empty list to collect values of `d` for each iteration
d_values =
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d = parts[2:4]
# Append `d` to your list, creating a list of lists
d_values.append(d)
# Once your loop is done, create a DataFrame from your list of lists
df = pd.DataFrame(d_values, columns=['col1name','col2name'])
answered Nov 12 at 1:30
dmitriys
15119
Thank you so much! really appreciate your help!
– tabarneck
Nov 12 at 4:08
add a comment |
1
1
1
I think the issue is that for each iteration of the loop you're creating a new DataFrame; whereas what I imagine you probably want is a single DataFrame with all of the values for d. Try the below:
# Instantiate an empty list to collect values of `d` for each iteration
d_values =
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d = parts[2:4]
# Append `d` to your list, creating a list of lists
d_values.append(d)
# Once your loop is done, create a DataFrame from your list of lists
df = pd.DataFrame(d_values, columns=['col1name','col2name'])
answered Nov 12 at 1:30
dmitriys
15119
I think the issue is that for each iteration of the loop you're creating a new DataFrame; whereas what I imagine you probably want is a single DataFrame with all of the values for d. Try the below:
# Instantiate an empty list to collect values of `d` for each iteration
d_values =
soup=BeautifulSoup(s,'lxml')
soup.prettify()
for anchor in soup.findAll('a', href=True):
s=anchor['href']
if 'bus' in s:
urlparse(s)
path = urlparse(s).path[1:]
parts = path.split('/')
d = parts[2:4]
# Append `d` to your list, creating a list of lists
d_values.append(d)
# Once your loop is done, create a DataFrame from your list of lists
df = pd.DataFrame(d_values, columns=['col1name','col2name'])
answered Nov 12 at 1:30
dmitriys
15119
answered Nov 12 at 1:30
dmitriys
15119
answered Nov 12 at 1:30
dmitriys
15119
answered Nov 12 at 1:30
dmitriys
15119
15119
Thank you so much! really appreciate your help!
– tabarneck
Nov 12 at 4:08
add a comment |
Thank you so much! really appreciate your help!
– tabarneck
Nov 12 at 4:08
Thank you so much! really appreciate your help!
– tabarneck
Nov 12 at 4:08
Thank you so much! really appreciate your help!
– tabarneck
Nov 12 at 4:08
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53254297%2fdump-of-lists-into-csv-export%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
Please replace the screenshots with the actual data. Screenshots of data, code, and errors do not belong on SO.
– DYZ
Nov 11 at 23:35
@DYZ but I am unable to get the data to csv hence why I am asking the question. I cannot copy paste the results for some reason from the command prompt. Its a lot of data by the way and its just a small fraction of it.
– tabarneck
Nov 11 at 23:47
What type of object is
d? Why are you passing it topd.DataFrameas a list (e.g. in). Have you triedpd.DataFrame(d)?– dmitriys
Nov 11 at 23:50
I get something like this if I do without the alon-De-Coiffure-Pour-Hommes-Laurier 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 King-s-Barbershop 0 0 Longueuil 1 Kings-Barbershop 0 0 Longueuil 1 Kings-Barbershop
– tabarneck
Nov 11 at 23:52
1
I will add the code to the post
– tabarneck
Nov 11 at 23:54