Getting data from an API

TL;DR#

Tonight we chose data structures by the shape of the data, wrote our first functions with def, and asked a server for data with requests. The habit worth keeping: read what came back before you use it. Check the outermost bracket, the status code, and whether the first result is the place you meant.

What we covered#

Commit from GitHub Desktop, and pull before you start. A file uploaded on github.com puts a change in your repository that your computer does not have. If you did that, click Fetch origin in GitHub Desktop, then Pull origin, before you change anything locally.

A list is read by position, a dictionary by key. orders[0] picks the first record, and orders[0]["ShipCountry"] picks one value from it. orders["ShipCountry"] stops with TypeError: list indices must be integers or slices, not str, because orders is a list. A key with the wrong capitals, such as "orderID", stops with KeyError.

A loop collects values, and set() keeps each one once. To list every ship country, start an empty list and append each order's "ShipCountry" in a for loop. set(countries) gives the distinct countries, and len(set(countries)) counts them.

in answers True or False. "China" in countries is True when the value is in the list. It works like a comparison, so it can go after if. In for order in orders: the same word is part of the loop.

Choose a structure by the shape of the data. The questions to ask are in notebook 3, section 4, and in Python §19. The customer rearranges the lines on one order by position, and moving an item to another position is what a list does. A dictionary keeps the order its keys were added in, but it has no position to move an entry to.

Read nested data one bracket at a time. Look at the outermost bracket first: [ is a list, { is a dictionary. Then take one step, check what you have with type(), and take the next. A list that holds a single dictionary still needs [0] before a key will work.

Write the algorithm in words first. For the portfolio: start a total at zero; for each holding, multiply shares by price; add that to the total. Each line of that plan becomes a line of Python.

Unpacking names each position. for symbol, shares, price in holdings: gives the three values of each tuple their own names, so the loop never needs holding[1]. The number of names has to match the number of values. A star takes whatever is left: area_code, *rest = "617-555-0123".split("-") gives "617" and ["555", "0123"].

A function is written once and changed in one place. def add_tax(amount): names the function and its parameter, and return hands back the result, the way a spreadsheet formula takes cells in and gives a value back. If the tax rate changes, you edit one line, and every call uses the new rate.

Larger programs are small functions put together. check_bmi(bmi) returns a category, calculate_bmi(height, weight) returns a number, and main() asks for height and weight and calls both. Each function has a clear input and output, so main() reads like the plan. Breaking a problem down this way is decomposition, and using a function by what it takes and returns, without reading its inside, is abstraction. When you ask an agent for a function, state its input and its output.

An API is an address that hands back data. requests.get(url) visits the address the way a browser does. .status_code says whether the server understood the request, and .json() turns the reply into Python lists and dictionaries. The 10m in wind_speed_10m is the height of the measurement: the wind 10 meters above the ground, the standard height for weather stations.

Read the reply as well as the status code. A latitude of 422 got 400, with a reply that says why. The misspelled Wellesly got 200 and a reply with no results in it.

Check that the first result is the place you meant. Without count=1, Open-Meteo's town search returns 10 places, and count= goes up to 100. A search for Wellesley returns places in Massachusetts, Ontario and Penang. The first Babson Park is not the one in Massachusetts, so code that always takes [0] can answer for the wrong town.

Turning a repository into a website#

GitHub Pages serves a repository as a website, at https://<your-github-username>.github.io/<repository>/. In class, an agent turned the lightning talk's PDF into a web page in a few minutes, and Pages put it online:

1. On github.com, open the repository, then Settings, then Pages
2. Under Build and deployment, choose Deploy from a branch, the branch main, and a folder, then Save
3. A file named index.html in that folder becomes the home page. A page appears a minute or two after each push

The demonstration used the root folder. To publish a marimo notebook, follow Publishing a Notebook, which uses the docs folder. A repository serves one folder at a time.

Public links#

• Session page
• Mini Project 2, the instructions and the four projects to choose from
• Course API documentation, every endpoint notebook 4 uses, with Try it out
• Publishing a Notebook, GitHub Pages for a marimo notebook
• Python handbook, Collections, for choosing a data structure

Before next session#

There will be no class on Tuesday 10/13. The graduate calendar treats that day as a Monday. We will meet next on Tuesday 10/20.

1. Mini Project 1 is due Sunday 10/11.
2. Start Mini Project 2 once Mini Project 1 is in. It is due Sunday 10/25. Choose a project, write your plan, and commit it before you ask your agent anything. The instructions are on the Mini Project 2 page.
3. Notebook 3, sections 4 to 6, 03-collections-and-apis.py: C and D on choosing a structure, and E to G on the weather.
4. Notebook 4, sections 1 to 3, 04-functions-and-apis.py: functions, then the course API, including every page of the towns. We will do section 4, sending data to the board, together on 10/20.
5. Commit and push as you go. Nothing counts until it is on github.com. Both notebooks are on Checkpoint 3, which closes Sunday 10/25.