In this post, we will see how to resolve Scraping data off Morningstar – Portfolio Screen
Question:I am trying to scrape data from this link https://www.morningstar.com/funds/xnas/gibix/portfolio — basically all the data I can get, but particularly the Fixed Income Style Table and the Exposure, Bond Breakdown table.
Here is my code:
'component'field in my header should be. I have tried even
'sal-components-fixed-income-exposure-analysis'but to no avail.
Best Answer:What you are doing is not web scraping, but an API request. There’s probably a way to get the data you want through the API but you might have to discover it from their docs: https://developer.morningstar.com/developer-resources/api-visualization-library/about
But I can provide you with a code snippet for actually scraping the data from this page:
To install lxml module just
pip install lxml
Ps: I tried getting the html with a request response but it’s returning another page, looks like you gotta open the page and wait until it’s fully loaded to get the correct source html.
If you have better answer, please add a comment about this, thank you!