Beautiful Soup4 'NoneType' object has no attribute 'text' Error
0
Tried scraping some numbers from Yahoo Finance, but I keep on getting an AttributeError for one of my soup.find functions. Code:
from bs4 import BeautifulSoup as soup
import requests
#Switch
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin- srch').text
soup = soup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
downorup = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)".split())))
print("Switch stock: " + price.text + ' ' + downorup.text)
python python-3.x beautifulsoup
asked Nov 11 at 6:26
R.Vij
176
|
show 1 more comment
0
Tried scraping some numbers from Yahoo Finance, but I keep on getting an AttributeError for one of my soup.find functions. Code:
from bs4 import BeautifulSoup as soup
import requests
#Switch
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin- srch').text
soup = soup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
downorup = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)".split())))
print("Switch stock: " + price.text + ' ' + downorup.text)
python python-3.x beautifulsoup
asked Nov 11 at 6:26
R.Vij
176
downorupis ` None. There is no span with the attributes that you requested. You must check a value returned byfind` before using it.
– DYZ
Nov 11 at 6:33
@JaydeepPatel Totally irrelevant.
– DYZ
Nov 11 at 6:33
There is a span: <span class="Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)" data-reactid="36">-0.43 (-4.42%)</span
– R.Vij
Nov 11 at 6:36
@JaydeepPatel Wrong,requests.get(url).textgives the HTML code (did you try?). Therequestsmodule does not know how to parse HTML.
– DYZ
Nov 11 at 6:42
@JaydeepPatel I tried using .content instead of .text, but I nevertheless get the same error.
– R.Vij
Nov 11 at 6:42
|
show 1 more comment
0
0
0
Tried scraping some numbers from Yahoo Finance, but I keep on getting an AttributeError for one of my soup.find functions. Code:
from bs4 import BeautifulSoup as soup
import requests
#Switch
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin- srch').text
soup = soup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
downorup = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)".split())))
print("Switch stock: " + price.text + ' ' + downorup.text)
python python-3.x beautifulsoup
asked Nov 11 at 6:26
R.Vij
176
Tried scraping some numbers from Yahoo Finance, but I keep on getting an AttributeError for one of my soup.find functions. Code:
from bs4 import BeautifulSoup as soup
import requests
#Switch
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin- srch').text
soup = soup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
downorup = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)".split())))
print("Switch stock: " + price.text + ' ' + downorup.text)
python python-3.x beautifulsoup
python python-3.x beautifulsoup
asked Nov 11 at 6:26
R.Vij
176
asked Nov 11 at 6:26
R.Vij
176
asked Nov 11 at 6:26
R.Vij
176
asked Nov 11 at 6:26
R.Vij
176
asked Nov 11 at 6:26
R.Vij
176
176
downorupis ` None. There is no span with the attributes that you requested. You must check a value returned byfind` before using it.
– DYZ
Nov 11 at 6:33
@JaydeepPatel Totally irrelevant.
– DYZ
Nov 11 at 6:33
There is a span: <span class="Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)" data-reactid="36">-0.43 (-4.42%)</span
– R.Vij
Nov 11 at 6:36
@JaydeepPatel Wrong,requests.get(url).textgives the HTML code (did you try?). Therequestsmodule does not know how to parse HTML.
– DYZ
Nov 11 at 6:42
@JaydeepPatel I tried using .content instead of .text, but I nevertheless get the same error.
– R.Vij
Nov 11 at 6:42
|
show 1 more comment
downorupis ` None. There is no span with the attributes that you requested. You must check a value returned byfind` before using it.
– DYZ
Nov 11 at 6:33
@JaydeepPatel Totally irrelevant.
– DYZ
Nov 11 at 6:33
There is a span: <span class="Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)" data-reactid="36">-0.43 (-4.42%)</span
– R.Vij
Nov 11 at 6:36
@JaydeepPatel Wrong,requests.get(url).textgives the HTML code (did you try?). Therequestsmodule does not know how to parse HTML.
– DYZ
Nov 11 at 6:42
@JaydeepPatel I tried using .content instead of .text, but I nevertheless get the same error.
– R.Vij
Nov 11 at 6:42
downorup is ` None. There is no span with the attributes that you requested. You must check a value returned by find` before using it.– DYZ
Nov 11 at 6:33
downorup is ` None. There is no span with the attributes that you requested. You must check a value returned by find` before using it.– DYZ
Nov 11 at 6:33
@JaydeepPatel Totally irrelevant.
– DYZ
Nov 11 at 6:33
@JaydeepPatel Totally irrelevant.
– DYZ
Nov 11 at 6:33
There is a span: <span class="Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)" data-reactid="36">-0.43 (-4.42%)</span
– R.Vij
Nov 11 at 6:36
There is a span: <span class="Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)" data-reactid="36">-0.43 (-4.42%)</span
– R.Vij
Nov 11 at 6:36
@JaydeepPatel Wrong,
requests.get(url).text gives the HTML code (did you try?). The requests module does not know how to parse HTML.– DYZ
Nov 11 at 6:42
@JaydeepPatel Wrong,
requests.get(url).text gives the HTML code (did you try?). The requests module does not know how to parse HTML.– DYZ
Nov 11 at 6:42
@JaydeepPatel I tried using .content instead of .text, but I nevertheless get the same error.
– R.Vij
Nov 11 at 6:42
@JaydeepPatel I tried using .content instead of .text, but I nevertheless get the same error.
– R.Vij
Nov 11 at 6:42
|
show 1 more comment
2 Answers
2
active
oldest
votes
1
In the second query, "Fz(24px)" must be "Fz(14px)", and there must be no "Pstart(10px)" part:
downorup = soup.find('span', class_=lambda x:x and set(x.split())
.issuperset(set("Trsdu(0.3s) Fw(500) Fz(14px) C($dataRed)".split())))
In general, you are using quite a dangerous approach to identify your data. If the formatting of the page changes, you won't find the fields anymore. "data-reactid" seems to be a more reliable marker.
answered Nov 11 at 6:47
DYZ
25.3k61948
It worked, but what was your logic? Why did I have to change the Fz() and Pstart()?
– R.Vij
Nov 11 at 6:50
All theseclassattributes are not about the data but about its presentation (formatting). Consider searching for data-reactid markers instead.
– DYZ
Nov 11 at 6:51
I cannot answer a question posted as a comment to my answer. I cannot even understand your code without indentation.
– DYZ
Nov 11 at 6:56
I tried again, this time with Costco (Which has a stock of 237.74), but get the same stock that Switch has as the result. Why?
– R.Vij
Nov 11 at 7:14
On the linesoup = soup(source, 'lxml'), you destroyed a reference to thesoupfunction (note the same name on the left and on the right). Your secondsoupwas not decoded, you probably got an error message.
– DYZ
Nov 11 at 7:18
|
show 4 more comments
0
The error is thrown most probably at the print statement.
If the code is not able to find price / downorup - it will have a value None.
In that case you are trying to access attributes price.text, downorup.text from a None object which throws the exception.
Add a check for None like:
if price and downorup:
print("Switch stock: " + price.text + ' ' + downorup.text)
This will go to print statement only if both price and downorup is not None.
answered Nov 11 at 6:34
justanotherguy
799
I tried but still got the same error. My error is not that I get "None" as downorup, it's that 'NoneType' has to attribute 'text'.
– R.Vij
Nov 11 at 6:46
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53246382%2fbeautiful-soup4-nonetype-object-has-no-attribute-text-error%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
In the second query, "Fz(24px)" must be "Fz(14px)", and there must be no "Pstart(10px)" part:
downorup = soup.find('span', class_=lambda x:x and set(x.split())
.issuperset(set("Trsdu(0.3s) Fw(500) Fz(14px) C($dataRed)".split())))
In general, you are using quite a dangerous approach to identify your data. If the formatting of the page changes, you won't find the fields anymore. "data-reactid" seems to be a more reliable marker.
answered Nov 11 at 6:47
DYZ
25.3k61948
It worked, but what was your logic? Why did I have to change the Fz() and Pstart()?
– R.Vij
Nov 11 at 6:50
All theseclassattributes are not about the data but about its presentation (formatting). Consider searching for data-reactid markers instead.
– DYZ
Nov 11 at 6:51
I cannot answer a question posted as a comment to my answer. I cannot even understand your code without indentation.
– DYZ
Nov 11 at 6:56
I tried again, this time with Costco (Which has a stock of 237.74), but get the same stock that Switch has as the result. Why?
– R.Vij
Nov 11 at 7:14
On the linesoup = soup(source, 'lxml'), you destroyed a reference to thesoupfunction (note the same name on the left and on the right). Your secondsoupwas not decoded, you probably got an error message.
– DYZ
Nov 11 at 7:18
|
show 4 more comments
1
In the second query, "Fz(24px)" must be "Fz(14px)", and there must be no "Pstart(10px)" part:
downorup = soup.find('span', class_=lambda x:x and set(x.split())
.issuperset(set("Trsdu(0.3s) Fw(500) Fz(14px) C($dataRed)".split())))
In general, you are using quite a dangerous approach to identify your data. If the formatting of the page changes, you won't find the fields anymore. "data-reactid" seems to be a more reliable marker.
answered Nov 11 at 6:47
DYZ
25.3k61948
It worked, but what was your logic? Why did I have to change the Fz() and Pstart()?
– R.Vij
Nov 11 at 6:50
All theseclassattributes are not about the data but about its presentation (formatting). Consider searching for data-reactid markers instead.
– DYZ
Nov 11 at 6:51
I cannot answer a question posted as a comment to my answer. I cannot even understand your code without indentation.
– DYZ
Nov 11 at 6:56
I tried again, this time with Costco (Which has a stock of 237.74), but get the same stock that Switch has as the result. Why?
– R.Vij
Nov 11 at 7:14
On the linesoup = soup(source, 'lxml'), you destroyed a reference to thesoupfunction (note the same name on the left and on the right). Your secondsoupwas not decoded, you probably got an error message.
– DYZ
Nov 11 at 7:18
|
show 4 more comments
1
1
1
In the second query, "Fz(24px)" must be "Fz(14px)", and there must be no "Pstart(10px)" part:
downorup = soup.find('span', class_=lambda x:x and set(x.split())
.issuperset(set("Trsdu(0.3s) Fw(500) Fz(14px) C($dataRed)".split())))
In general, you are using quite a dangerous approach to identify your data. If the formatting of the page changes, you won't find the fields anymore. "data-reactid" seems to be a more reliable marker.
answered Nov 11 at 6:47
DYZ
25.3k61948
In the second query, "Fz(24px)" must be "Fz(14px)", and there must be no "Pstart(10px)" part:
downorup = soup.find('span', class_=lambda x:x and set(x.split())
.issuperset(set("Trsdu(0.3s) Fw(500) Fz(14px) C($dataRed)".split())))
In general, you are using quite a dangerous approach to identify your data. If the formatting of the page changes, you won't find the fields anymore. "data-reactid" seems to be a more reliable marker.
answered Nov 11 at 6:47
DYZ
25.3k61948
answered Nov 11 at 6:47
DYZ
25.3k61948
answered Nov 11 at 6:47
DYZ
25.3k61948
answered Nov 11 at 6:47
DYZ
25.3k61948
25.3k61948
It worked, but what was your logic? Why did I have to change the Fz() and Pstart()?
– R.Vij
Nov 11 at 6:50
All theseclassattributes are not about the data but about its presentation (formatting). Consider searching for data-reactid markers instead.
– DYZ
Nov 11 at 6:51
I cannot answer a question posted as a comment to my answer. I cannot even understand your code without indentation.
– DYZ
Nov 11 at 6:56
I tried again, this time with Costco (Which has a stock of 237.74), but get the same stock that Switch has as the result. Why?
– R.Vij
Nov 11 at 7:14
On the linesoup = soup(source, 'lxml'), you destroyed a reference to thesoupfunction (note the same name on the left and on the right). Your secondsoupwas not decoded, you probably got an error message.
– DYZ
Nov 11 at 7:18
|
show 4 more comments
It worked, but what was your logic? Why did I have to change the Fz() and Pstart()?
– R.Vij
Nov 11 at 6:50
All theseclassattributes are not about the data but about its presentation (formatting). Consider searching for data-reactid markers instead.
– DYZ
Nov 11 at 6:51
I cannot answer a question posted as a comment to my answer. I cannot even understand your code without indentation.
– DYZ
Nov 11 at 6:56
I tried again, this time with Costco (Which has a stock of 237.74), but get the same stock that Switch has as the result. Why?
– R.Vij
Nov 11 at 7:14
On the linesoup = soup(source, 'lxml'), you destroyed a reference to thesoupfunction (note the same name on the left and on the right). Your secondsoupwas not decoded, you probably got an error message.
– DYZ
Nov 11 at 7:18
It worked, but what was your logic? Why did I have to change the Fz() and Pstart()?
– R.Vij
Nov 11 at 6:50
It worked, but what was your logic? Why did I have to change the Fz() and Pstart()?
– R.Vij
Nov 11 at 6:50
All these
class attributes are not about the data but about its presentation (formatting). Consider searching for data-reactid markers instead.– DYZ
Nov 11 at 6:51
All these
class attributes are not about the data but about its presentation (formatting). Consider searching for data-reactid markers instead.– DYZ
Nov 11 at 6:51
I cannot answer a question posted as a comment to my answer. I cannot even understand your code without indentation.
– DYZ
Nov 11 at 6:56
I cannot answer a question posted as a comment to my answer. I cannot even understand your code without indentation.
– DYZ
Nov 11 at 6:56
I tried again, this time with Costco (Which has a stock of 237.74), but get the same stock that Switch has as the result. Why?
– R.Vij
Nov 11 at 7:14
I tried again, this time with Costco (Which has a stock of 237.74), but get the same stock that Switch has as the result. Why?
– R.Vij
Nov 11 at 7:14
On the line
soup = soup(source, 'lxml'), you destroyed a reference to the soup function (note the same name on the left and on the right). Your second soup was not decoded, you probably got an error message.– DYZ
Nov 11 at 7:18
On the line
soup = soup(source, 'lxml'), you destroyed a reference to the soup function (note the same name on the left and on the right). Your second soup was not decoded, you probably got an error message.– DYZ
Nov 11 at 7:18
|
show 4 more comments
0
The error is thrown most probably at the print statement.
If the code is not able to find price / downorup - it will have a value None.
In that case you are trying to access attributes price.text, downorup.text from a None object which throws the exception.
Add a check for None like:
if price and downorup:
print("Switch stock: " + price.text + ' ' + downorup.text)
This will go to print statement only if both price and downorup is not None.
answered Nov 11 at 6:34
justanotherguy
799
I tried but still got the same error. My error is not that I get "None" as downorup, it's that 'NoneType' has to attribute 'text'.
– R.Vij
Nov 11 at 6:46
add a comment |
0
The error is thrown most probably at the print statement.
If the code is not able to find price / downorup - it will have a value None.
In that case you are trying to access attributes price.text, downorup.text from a None object which throws the exception.
Add a check for None like:
if price and downorup:
print("Switch stock: " + price.text + ' ' + downorup.text)
This will go to print statement only if both price and downorup is not None.
answered Nov 11 at 6:34
justanotherguy
799
I tried but still got the same error. My error is not that I get "None" as downorup, it's that 'NoneType' has to attribute 'text'.
– R.Vij
Nov 11 at 6:46
add a comment |
0
0
0
The error is thrown most probably at the print statement.
If the code is not able to find price / downorup - it will have a value None.
In that case you are trying to access attributes price.text, downorup.text from a None object which throws the exception.
Add a check for None like:
if price and downorup:
print("Switch stock: " + price.text + ' ' + downorup.text)
This will go to print statement only if both price and downorup is not None.
answered Nov 11 at 6:34
justanotherguy
799
The error is thrown most probably at the print statement.
If the code is not able to find price / downorup - it will have a value None.
In that case you are trying to access attributes price.text, downorup.text from a None object which throws the exception.
Add a check for None like:
if price and downorup:
print("Switch stock: " + price.text + ' ' + downorup.text)
This will go to print statement only if both price and downorup is not None.
answered Nov 11 at 6:34
justanotherguy
799
answered Nov 11 at 6:34
justanotherguy
799
answered Nov 11 at 6:34
justanotherguy
799
answered Nov 11 at 6:34
justanotherguy
799
799
I tried but still got the same error. My error is not that I get "None" as downorup, it's that 'NoneType' has to attribute 'text'.
– R.Vij
Nov 11 at 6:46
add a comment |
I tried but still got the same error. My error is not that I get "None" as downorup, it's that 'NoneType' has to attribute 'text'.
– R.Vij
Nov 11 at 6:46
I tried but still got the same error. My error is not that I get "None" as downorup, it's that 'NoneType' has to attribute 'text'.
– R.Vij
Nov 11 at 6:46
I tried but still got the same error. My error is not that I get "None" as downorup, it's that 'NoneType' has to attribute 'text'.
– R.Vij
Nov 11 at 6:46
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53246382%2fbeautiful-soup4-nonetype-object-has-no-attribute-text-error%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
downorupis ` None. There is no span with the attributes that you requested. You must check a value returned byfind` before using it.– DYZ
Nov 11 at 6:33
@JaydeepPatel Totally irrelevant.
– DYZ
Nov 11 at 6:33
There is a span: <span class="Trsdu(0.3s) Fw(500) Pstart(10px) Fz(24px) C($dataRed)" data-reactid="36">-0.43 (-4.42%)</span
– R.Vij
Nov 11 at 6:36
@JaydeepPatel Wrong,
requests.get(url).textgives the HTML code (did you try?). Therequestsmodule does not know how to parse HTML.– DYZ
Nov 11 at 6:42
@JaydeepPatel I tried using .content instead of .text, but I nevertheless get the same error.
– R.Vij
Nov 11 at 6:42