BeautifulSoup4 Find Method
I tried scraping some a number from yahoo finance using python3, but all I get is a "None".
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?
p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)')
print(price)
Thanks,
R.Vij
python python-3.x beautifulsoup
asked Nov 11 at 2:48
R.Vij
176
add a comment |
I tried scraping some a number from yahoo finance using python3, but all I get is a "None".
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?
p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)')
print(price)
Thanks,
R.Vij
python python-3.x beautifulsoup
asked Nov 11 at 2:48
R.Vij
176
add a comment |
I tried scraping some a number from yahoo finance using python3, but all I get is a "None".
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?
p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)')
print(price)
Thanks,
R.Vij
python python-3.x beautifulsoup
asked Nov 11 at 2:48
R.Vij
176
I tried scraping some a number from yahoo finance using python3, but all I get is a "None".
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?
p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)')
print(price)
Thanks,
R.Vij
python python-3.x beautifulsoup
python python-3.x beautifulsoup
asked Nov 11 at 2:48
R.Vij
176
asked Nov 11 at 2:48
R.Vij
176
edited Nov 11 at 2:54
asked Nov 11 at 2:48
R.Vij
176
asked Nov 11 at 2:48
R.Vij
176
asked Nov 11 at 2:48
R.Vij
176
176
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
Note that if you pass a list to the class_ kwarg bs4 will select elements that have ANY of the specified classNames in the document, not ALL of them.
Also you need to note that some of the class values are set dynamically using browser javascript so that they won't appear on the actual document.
I revised your find statement to the following one:
soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
The following code returns the current price of SWCH
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
print(price.text) # 9.29 for now
answered Nov 11 at 3:24
Eternal_flame-AD
3126
Thanks, it worked. Do you know how the lambda stuff works and what it does? I like to know what code does.
– R.Vij
Nov 11 at 3:53
Beautifulsoup4 would call the lambda function for every element with the value of the class attribute of that element(None if the class attribute does not exist) and select the element if the lambda function returns True. The lambda function checks if the class attribute exists(the firstx andstuff) and then checks if the actual class names are a superset of the required class names.
– Eternal_flame-AD
Nov 11 at 3:58
add a comment |
'Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)' is not a class but five classes. If you want to find any of them, you should pass them as a list:
soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)'.split())
#<span class="D(ib) W($privatePromoMsgWidth) Fz(12px) Fw(500) Va(m) Wob(n)"...
answered Nov 11 at 2:54
DYZ
25.3k61948
I tried, but what I got inside the tags was "Now you can search stock related news and private companies such as Airbnb." The result should be "9.29".
– R.Vij
Nov 11 at 3:48
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53245424%2fbeautifulsoup4-find-method%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Note that if you pass a list to the class_ kwarg bs4 will select elements that have ANY of the specified classNames in the document, not ALL of them.
Also you need to note that some of the class values are set dynamically using browser javascript so that they won't appear on the actual document.
I revised your find statement to the following one:
soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
The following code returns the current price of SWCH
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
print(price.text) # 9.29 for now
answered Nov 11 at 3:24
Eternal_flame-AD
3126
Thanks, it worked. Do you know how the lambda stuff works and what it does? I like to know what code does.
– R.Vij
Nov 11 at 3:53
Beautifulsoup4 would call the lambda function for every element with the value of the class attribute of that element(None if the class attribute does not exist) and select the element if the lambda function returns True. The lambda function checks if the class attribute exists(the firstx andstuff) and then checks if the actual class names are a superset of the required class names.
– Eternal_flame-AD
Nov 11 at 3:58
add a comment |
Note that if you pass a list to the class_ kwarg bs4 will select elements that have ANY of the specified classNames in the document, not ALL of them.
Also you need to note that some of the class values are set dynamically using browser javascript so that they won't appear on the actual document.
I revised your find statement to the following one:
soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
The following code returns the current price of SWCH
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
print(price.text) # 9.29 for now
answered Nov 11 at 3:24
Eternal_flame-AD
3126
Thanks, it worked. Do you know how the lambda stuff works and what it does? I like to know what code does.
– R.Vij
Nov 11 at 3:53
Beautifulsoup4 would call the lambda function for every element with the value of the class attribute of that element(None if the class attribute does not exist) and select the element if the lambda function returns True. The lambda function checks if the class attribute exists(the firstx andstuff) and then checks if the actual class names are a superset of the required class names.
– Eternal_flame-AD
Nov 11 at 3:58
add a comment |
Note that if you pass a list to the class_ kwarg bs4 will select elements that have ANY of the specified classNames in the document, not ALL of them.
Also you need to note that some of the class values are set dynamically using browser javascript so that they won't appear on the actual document.
I revised your find statement to the following one:
soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
The following code returns the current price of SWCH
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
print(price.text) # 9.29 for now
answered Nov 11 at 3:24
Eternal_flame-AD
3126
Note that if you pass a list to the class_ kwarg bs4 will select elements that have ANY of the specified classNames in the document, not ALL of them.
Also you need to note that some of the class values are set dynamically using browser javascript so that they won't appear on the actual document.
I revised your find statement to the following one:
soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
The following code returns the current price of SWCH
from bs4 import BeautifulSoup
import requests
source = requests.get('https://finance.yahoo.com/quote/SWCH?p=SWCH&.tsrc=fin-srch').text
soup = BeautifulSoup(source, 'lxml')
price = soup.find('span', class_=lambda x:x and set(x.split()).issuperset(set("Trsdu(0.3s) Fw(b) Fz(36px) Fw(b) D(b) Mb(-4px)".split())))
print(price.text) # 9.29 for now
answered Nov 11 at 3:24
Eternal_flame-AD
3126
answered Nov 11 at 3:24
Eternal_flame-AD
3126
answered Nov 11 at 3:24
Eternal_flame-AD
3126
answered Nov 11 at 3:24
Eternal_flame-AD
3126
3126
Thanks, it worked. Do you know how the lambda stuff works and what it does? I like to know what code does.
– R.Vij
Nov 11 at 3:53
Beautifulsoup4 would call the lambda function for every element with the value of the class attribute of that element(None if the class attribute does not exist) and select the element if the lambda function returns True. The lambda function checks if the class attribute exists(the firstx andstuff) and then checks if the actual class names are a superset of the required class names.
– Eternal_flame-AD
Nov 11 at 3:58
add a comment |
Thanks, it worked. Do you know how the lambda stuff works and what it does? I like to know what code does.
– R.Vij
Nov 11 at 3:53
Beautifulsoup4 would call the lambda function for every element with the value of the class attribute of that element(None if the class attribute does not exist) and select the element if the lambda function returns True. The lambda function checks if the class attribute exists(the firstx andstuff) and then checks if the actual class names are a superset of the required class names.
– Eternal_flame-AD
Nov 11 at 3:58
Thanks, it worked. Do you know how the lambda stuff works and what it does? I like to know what code does.
– R.Vij
Nov 11 at 3:53
Thanks, it worked. Do you know how the lambda stuff works and what it does? I like to know what code does.
– R.Vij
Nov 11 at 3:53
Beautifulsoup4 would call the lambda function for every element with the value of the class attribute of that element(None if the class attribute does not exist) and select the element if the lambda function returns True. The lambda function checks if the class attribute exists(the first
x and stuff) and then checks if the actual class names are a superset of the required class names.– Eternal_flame-AD
Nov 11 at 3:58
Beautifulsoup4 would call the lambda function for every element with the value of the class attribute of that element(None if the class attribute does not exist) and select the element if the lambda function returns True. The lambda function checks if the class attribute exists(the first
x and stuff) and then checks if the actual class names are a superset of the required class names.– Eternal_flame-AD
Nov 11 at 3:58
add a comment |
'Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)' is not a class but five classes. If you want to find any of them, you should pass them as a list:
soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)'.split())
#<span class="D(ib) W($privatePromoMsgWidth) Fz(12px) Fw(500) Va(m) Wob(n)"...
answered Nov 11 at 2:54
DYZ
25.3k61948
I tried, but what I got inside the tags was "Now you can search stock related news and private companies such as Airbnb." The result should be "9.29".
– R.Vij
Nov 11 at 3:48
add a comment |
'Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)' is not a class but five classes. If you want to find any of them, you should pass them as a list:
soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)'.split())
#<span class="D(ib) W($privatePromoMsgWidth) Fz(12px) Fw(500) Va(m) Wob(n)"...
answered Nov 11 at 2:54
DYZ
25.3k61948
I tried, but what I got inside the tags was "Now you can search stock related news and private companies such as Airbnb." The result should be "9.29".
– R.Vij
Nov 11 at 3:48
add a comment |
'Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)' is not a class but five classes. If you want to find any of them, you should pass them as a list:
soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)'.split())
#<span class="D(ib) W($privatePromoMsgWidth) Fz(12px) Fw(500) Va(m) Wob(n)"...
answered Nov 11 at 2:54
DYZ
25.3k61948
'Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)' is not a class but five classes. If you want to find any of them, you should pass them as a list:
soup.find('span', class_='Trsdu(0.3s) Fw(b) Fz(36px) Mb(-4px) D(ib)'.split())
#<span class="D(ib) W($privatePromoMsgWidth) Fz(12px) Fw(500) Va(m) Wob(n)"...
answered Nov 11 at 2:54
DYZ
25.3k61948
answered Nov 11 at 2:54
DYZ
25.3k61948
answered Nov 11 at 2:54
DYZ
25.3k61948
answered Nov 11 at 2:54
DYZ
25.3k61948
25.3k61948
I tried, but what I got inside the tags was "Now you can search stock related news and private companies such as Airbnb." The result should be "9.29".
– R.Vij
Nov 11 at 3:48
add a comment |
I tried, but what I got inside the tags was "Now you can search stock related news and private companies such as Airbnb." The result should be "9.29".
– R.Vij
Nov 11 at 3:48
I tried, but what I got inside the tags was "Now you can search stock related news and private companies such as Airbnb." The result should be "9.29".
– R.Vij
Nov 11 at 3:48
I tried, but what I got inside the tags was "Now you can search stock related news and private companies such as Airbnb." The result should be "9.29".
– R.Vij
Nov 11 at 3:48
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53245424%2fbeautifulsoup4-find-method%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
