9.1 From Pyret to Python

Through our work in Pyret to this point, we’ve covered several core programming skills: how to work with tables, how to design good examples, the basics of creating datatypes, and how to work with the fundamental computational building blocks of functions, conditionals, and repetition (through filter and map, as well as recursion). You’ve got a solid initial toolkit, as well as a wide world of other possible programs ahead of you!

But we’re going to shift gears for a little while and show you how to work in Python instead. Why?

Seeing how the same concepts play out in multiple languages can help you distinguish core computational ideas from the notations and idioms of specific languages. If you plan to write programs as part of your professional work, you’ll inevitably have to work in different languages at different times: we’re giving you a chance to practice that skill in a controlled and gentle setting.

Why do we call this gentle? Because the notations in Pyret were designed partly with this transition in mind. You’ll find many similarities between Pyret and Python at a notational level, yet also some interesting differences that highlight some philosophical differences that underlie languages. The next set of programs that we want to write (specifically, data-rich programs where the data must be updated and maintained over time) fit nicely with certain features of Python that you haven’t seen in Pyret. A future release will contain material that contrasts the strengths and weaknesses of the two languages.

We highlight the basic notational differences between Pyret and Python by redoing some of our earlier code examples in Python.

9.1.1 Expressions, Functions, and Types

Back in Functions Practice: Cost of pens, we introduced the notation for functions and types using an example of computing the cost of an order of pens. An order consisted of a number of pens and a message to be printed on the pens. Each pen cost 25 cents, plus 2 cents per character for the message. Here was the original Pyret code:

fun pen-cost(num-pens :: Number, message :: String) -> Number:
  doc: ```total cost for pens, each 25 cents
          plus 2 cents per message character```
  num-pens * (0.25 + (string-length(message) * 0.02))
end

Here’s the corresponding Python code:

def pen_cost(num_pens: int, message: str) -> float:
    """total cost for pens, each at 25 cents plus
       2 cents per message character"""
    return num_pens * (0.25 + (len(message) * 0.02))

Do Now!

What notational differences do you see between the two versions?

Here’s a summary of the differences:

• Python uses def instead of fun.

• Python uses underscores in names (like pen_cost) instead of hyphens as in Pyret.

• The type names are written differently: Python uses str and int instead of String and Number. In addition, Python uses only a single colon before the type whereas Pyret uses a double colon.

• Python has different types for different kinds of numbers: int is for integers, while float is for decimals. Pyret just used a single type (Number) for all numbers.

• Python doesn’t label the documentation string (as Pyret does with doc:).

• There is no end annotation in Python. Instead, Python uses indentation to locate the end of an if/else statement, function, or other multi-line construct.

• Python labels the outputs of functions with return.

These are minor differences in notation, which you will get used to as you write more programs in Python.

There are differences beyond the notational ones. One that arises with this sample program arises around how the language uses types. In Pyret, if you put a type annotation on a parameter then pass it a value of a different type, you’ll get an error message. Python ignores the type annotations (unless you bring in additional tools for checking types). Python types are like notes for programmers, but they aren’t enforced when programs run.

Exercise

Convert the following moon-weight function from Functions Practice: Moon Weight into Python:

fun moon-weight(earth-weight :: Number) -> Number:
  doc:" Compute weight on moon from weight on earth"
  earth-weight * 1/6
end

9.1.2 Returning Values from Functions

In Pyret, a function body consisted of optional statements to name intermediate values, followed by a single expression. The value of that single expression is the result of calling the function. In Pyret, every function produces a result, so there is no need to label where the result comes from.

As we will see, Python is different: not all “functions” return results (note the name change from fun to def).In mathematics, functions have results by definition. Programmers sometimes distinguish between the terms “function” and “procedure”: both refer to parameterized computations, but only the former returns a result to the surrounding computation. Some programmers and languages do, however, use the term “function” more loosely to cover both kinds of parameterized computations. Moreover, the result isn’t necessarily the last expression of the def. In Python, the keyword return explicitly labels the expression whose value serves as the result of the function.

Do Now!

Put these two definitions in a Python file.

def add1v1(x: int) -> int:
    return x + 1

def add1v2(x: int) -> int:
    x + 1

At the Python prompt, call each function in turn. What do you notice about the result from using each function?

Hopefully, you noticed that using add1v1 displays an answer after the prompt, while using add1v2 does not. This difference has consequences for composing functions.

Do Now!

Try evaluating the following two expressions at the Python prompt: what happens in each case?

3 * add1v1(4)

3 * add1v2(4)

This example illustrates why return is essential in Python: without it, no value is returned, which means you can’t use the result of a function within another expression. So what use is add1v2 then? Hold that question; we’ll return to it in Modifying Variables.

9.1.3 Examples and Test Cases

In Pyret, we included examples with every function using where: blocks. We also had the ability to write check: blocks for more extensive tests. As a reminder, here was the pen-cost code including a where: block:

fun pen-cost(num-pens :: Number, message :: String) -> Number:
  doc: ```total cost for pens, each 25 cents
       plus 2 cents per message character```
  num-pens * (0.25 + (string-length(message) * 0.02))
where:
  pen-cost(1, "hi") is 0.29
  pen-cost(10, "smile") is 3.50
end

Python does not have a notion of where: blocks, or a distinction between examples and tests. There are a couple of different testing packages for Python; here we will use pytest, a standard lightweight framework that resembles the form of testing that we did in Pyret.How you set up pytest and your test file contents will vary according to your Python IDE. We assume instructors will provide separate instructions that align with their tool choices. To use pytest, we put both examples and tests in a separate function. Here’s an example of this for the pen_cost function:

import pytest

def pen_cost(num_pens: int, message: str) -> float:
    """total cost for pens, each at 25 cents plus
       2 cents per message character"""
    return num_pens * (0.25 + (len(message) * 0.02))

def test_pens():
  assert pen_cost(1, "hi") == 0.29
  assert pen_cost(10, "smile") == 3.50

Things to note about this code:

• We’ve imported pytest, the lightweight Python testing library.

• The examples have moved into a function (here test_pens) that takes no inputs. Note that the names of functions that contain test cases must have names that start with test_ in order for pytest to find them.

• In Python, individual tests have the form

  assert EXPRESSION == EXPECTED_ANS

  rather than the is form from Pyret.

Do Now!

Add one more test to the Python code, corresponding to the Pyret test

pen-cost(3, "wow") is 0.93

Make sure to run the test.

Do Now!

Did you actually try to run the test?

Whoa! Something weird happened: the test failed. Stop and think about that: the same test that worked in Pyret failed in Python. How can that be?

9.1.4 An Aside on Numbers

It turns out that different programming languages make different decisions about how to represent and manage real (non-integer) numbers. Sometimes, differences in these representations lead to subtle quantitative differences in computed values. As a simple example, let’s look at two seemingly simple real numbers 1/2 and 1/3. Here’s what we get when we type these two numbers at a Pyret prompt:

0.5

0.3

If we type these same two numbers in a Python console, we instead get:

0.5

0.3333333333333333

Notice that the answers look different for 1/3. As you may (or may not!) recall from an earlier math class, 1/3 is an example of a non-terminating, repeating decimal. In plain terms, if we tried to write out the exact value of 1/3 in decimal form, we would need to write an infinite sequence of 3. Mathematicians denote this by putting a horizontal bar over the 3. This is the notation we see in Pyret. Python, in contrast, writes out a partial sequence of 3s.

Underneath this distinction lies some interesting details about representing numbers in computers. Computers don’t have infinite space to store numbers (or anything else, for that matter): when a program needs to work with a non-terminating decimal, the underlying language can either:

• approximate the number (by chopping off the infinite sequence of digits at some point), then work only with the approximated value going forward, or

• store additional information about the number that may enable doing more precise computation with it later (though there are always some numbers that cannot be represented exactly in finite space).

Python takes the first approach. As a result, computations with the approximated values sometimes yield approximated results. This is what happens with our new pen_cost test case. While mathematically, the computation should result in 0.93, the approximations yield 0.9299999999999999 instead.

So how do we write tests in this situation? We need to tell Python that the answer should be “close” to 0.93, within the error range of approximations. Here’s what that looks like:

assert pen_cost(3, "wow") == pytest.approx(0.93)

9.1.5 Conditionals

Continuing with our original pen_cost example, here’s the Python version of the function that computed shipping costs on an order:

def add_shipping(order_amt: float) -> float:
    """increase order price by costs for shipping"""
    if order_amt == 0:
      return 0
    elif order_amt <= 10:
      return order_amt + 4
    elif (order_amt > 10) and (order_amt < 30):
      return order_amt + 8
    else:
      return order_amt + 12

The main difference to notice here is that else if is written as the single-word elif in Python. We use return to mark the function’s results in each branch of the conditional. Otherwise, the conditional constructs are quite similar across the two languages.

You may have noticed that Python does not require an explicit end annotation on if-expressions or functions. Instead, Python looks at the indentation of your code to determine when a construct has ended. For example, in the code sample for pen_cost and test_pens, Python determines that the pen_cost function has ended because it detects a new definition (for test_pens) at the left edge of the program text. The same principle holds for ending conditionals.

We’ll return to this point about indentation, and see more examples, as we work more with Python.

9.1.6 Creating and Processing Lists

As an example of lists, let’s assume we’ve been playing a game that involves making words out of a collection of letters. In Pyret, we could have written a sample word list as follows:

words = [list: "banana", "bean", "falafel", "leaf"]

In Python, this definition would look like:

words = ["banana", "bean", "falafel", "leaf"]

The only difference here is that Python does not use the list: label that is needed in Pyret.

9.1.6.1 Filters, Maps, and Friends

When we first learned about lists in Pyret, we started with common built-in functions such as filter, map, member and length. We also saw the use of lambda to help us use some of these functions concisely. These same functions, including lambda, also exist in Python. Here are some samples (# is the comment character in Python):

words = ["banana", "bean", "falafel", "leaf"]

# filter and member
words_with_b = list(filter(lambda wd: "b" in wd, words))
# filter and length
short_words = list(filter(lambda wd: len(wd) < 5, words))
# map and length
word_lengths = list(map(len, words))

Note that you have to wrap calls to filter (and map) with a use of list(). Internally, Python has these functions return a type of data that we haven’t yet discussed (and don’t need). Using list converts the returned data into a list. If you omit the list, you won’t be able to chain certain functions together. For example, if we tried to compute the length of the result of a map without first converting to a list, we’d get an error:

TypeError: object of type 'map' has no len()

Don’t worry if this error message makes no sense at the moment (we haven’t yet learned what an “object” is). The point is that if you see an error like this while using the result of filter or map, you likely forgot to wrap the result in list.

Exercise

Practice Python’s list functions by writing expressions for the following problems. Use only the list functions we have shown you so far.

• Given a list of numbers, convert it to a list of strings "pos", "neg", "zero", based on the sign of each number.

• Given a list of strings, is the length of any string equal to 5?

• Given a list of numbers, produce a list of the even numbers between 10 and 20 from that list.

We’re intentionally focusing on computations that use Python’s built-in functions for processing lists, rather than showing you how to write you own (as we did with recursion in Pyret). While you can write recursive functions to process lists in Pyret, a different style of program is more conventional for that purpose. We’ll look at that in the chapter on Modifying Variables.

9.1.7 Data with Components

An analog to a Pyret data definition (without variants) is called a dataclass in Python.Those experienced with Python may wonder why we are using dataclasses instead of dictionaries or raw classes. Compared to dictionaries, dataclasses allow the use of type hints and capture that our data has a fixed collection of fields. Compared to raw classes, dataclasses generate a lot of boilerplate code that makes them much lighterweight than raw classes. Here’s an example of a todo-list datatype in Pyret and its corresponding Python code:

# a todo item in Pyret
data ToDoItemData:
  | todoItem(descr :: String,
             due :: Date,
             tags :: List<String>
end

------------------------------------------
# the same todo item in Python

# to allow use of dataclasses
from dataclasses import dataclass
# to allow dates as a type (in the ToDoItem)
from datetime import date

@dataclass
class ToDoItem:
    descr: str
    due: date
    tags: list

# a sample list of ToDoItem
myTD = [ToDoItem("buy milk", date(2020, 7, 27), ["shopping", "home"]),
        ToDoItem("grade hwk", date(2020, 7, 27), ["teaching"]),
        ToDoItem("meet students", date(2020, 7, 26), ["research"])
       ]

Things to note:

• There is a single name for the type and the constructor, rather than separate names as we had in Pyret.

• There are no commas between field names (but each has to be on its own line in Python)

• There is no way to specify the type of the contents of the list in Python (at least, not without using more advance packages for writing types)

• The @dataclass annotation is needed before class.

• Dataclasses don’t support creating datatypes with multiple variants, like we did frequently in Pyret. Doing that needs more advanced concepts than we will cover in this book.

9.1.7.1 Accessing Fields within Dataclasses

In Pyret, we extracted a field from structured data by using a dot (period) to “dig into” the datum and access the field. The same notation works in Python:

"buy tickets"

9.1.8 Traversing Lists

9.1.8.1 Introducing For Loops

fun sum-list(numlist :: List<Number>) -> Number:
  cases (List) numlist:
    | empty => 0
    | link(fst, rst) => fst + sum-list(rst)
  end
end

In Python, it is unusual to break a list into its first and rest components and process the rest recursively. Instead, we use a construct called a for to visit each element of a list in turn. Here’s the form of for, using a concrete (example) list of odd numbers:

for num in [5, 1, 7, 3]:
   # do something with num

# do something with 5
# do something with 1
# do something with 7
# do something with 3

Let’s now use for to compute the running sum of a list. We’ll start by figuring out the repeated computation with our concrete list again. At first, let’s express the repeated computation just in prose. In Pyret, our repeated computation was along the lines of “add the first item to the sum of the rest of the items”. We’ve already said that we cannot easily access the “rest of the items” in Python, so we need to rephrase this. Here’s an alternative:

# set a running total to 0
# add 5 to the running total
# add 1 to the running total
# add 7 to the running total
# add 3 to the running total

Let’s convert this prose sketch to code by replacing each line of the sketch with concrete code. We do this by setting up a variable named run_total and updating its value for each element.

run_total = 0
run_total = run_total + 5
run_total = run_total + 1
run_total = run_total + 7
run_total = run_total + 3

run_total = 0
for num in [5, 1, 7, 3]:
   run_total = run_total + num

def sum_list(numlist : list) -> float:
    """sum a list of numbers"""
    run_total = 0
    for num in numlist:
        run_total = run_total + num
    return(run_total)

Do Now!

Write a set of tests for sum_list (the Python version).

Now that the Python version is done, let’s compare it to the original Pyret version:

fun sum-list(numlist :: List<Number>) -> Number:
  cases (List) numlist:
    | empty => 0
    | link(fst, rst) => fst + sum-list(rst)
  end
end

Here are some things to notice about the two pieces of code:

• The Python version needs a variable (here run_total) to hold the result of the computation as we build it up while traversing (working through) the list.

• The initial value of that variable is the answer we returned in the empty case in Pyret.

• The computation in the link case of the Pyret function is used to update that variable in the body of the for.

• After the for has finished processing all items in the list, the Python version returns the value in the variable as the result of the function.

9.1.8.2 An Aside on Order of Processing List Elements

There’s another subtlety here if we consider how the two programs run: the Python version sums the elements from left to right, whereas the Pyret version sums them right to left. Concretely, the sequence of values of run_total are computed as:

run_total = 0
run_total = 0 + 5
run_total = 5 + 1
run_total = 6 + 7
run_total = 13 + 3

sum_list([list: 5, 1, 7, 3])
5 + sum_list([list: 1, 7, 3])
5 + 1 + sum_list([list: 7, 3])
5 + 1 + 7 + sum_list([list: 3])
5 + 1 + 7 + 3 + sum_list([list:])
5 + 1 + 7 + 3 + 0
5 + 1 + 7 + 3
5 + 1 + 10
5 + 11
16

In the case of summing a list, we don’t notice the difference between the two versions because the sum is the same whether we compute it left-to-right or right-to-left. In other functions we write, this difference may start to matter.

9.1.8.3 Using For Loops in Functions that Produce Lists

Let’s practice using for loops on another function that traverses lists, this time one that produces a list. Specifically, let’s write a program that takes a list of strings and produces a list of words within that list that contain the letter "z".

As in our sum_list function, we will need a variable to store the resulting list as we build it up. The following code calls this zlist. The code also shows how to use in to check whether a character is in a string (it also works for checking whether an item is in a list) and how to add an element to the end of a list (append).

def all_z_words(wordlist : list) -> list:
    """produce list of words from the input that contain z"""
    zlist = [] # start with an empty list
    for wd in wordlist:
        if "z" in wd:
            zlist = [wd] + zlist
    return(zlist)

Exercise

Write tests for all_z_words.

Exercise

Write a second version of all_z_words using filter. Be sure to write tests for it!

Exercise

Contrast these two versions and the corresponding tests. Did you notice anything interesting?

9.1.8.4 Summary: The List-Processing Template for Python

Just as we had a template for writing list-processing functions in Pyret, there is a corresponding template in Python based on for loops. As a reminder, that pattern is as follow:

def func(lst: list):
  result = ...  # what to return if the input list is empty
  for item in lst:
    # combine item with the result so far
    result = ... item ... result
  return result

Keep this template in mind as you learn to write functions over lists in Python.

9.1.8.5 for each loops in Pyret

This section can be read without reading the rest of this chapter, so if you have been directed to it before being introduced to Python, do not worry! While the content below mirrors similar constructs that exist in Python, it is introduced on its own.

The previous sections introduced for loops in Python, and showed a template for processing lists with them. Pyret can do similar, using the following pattern:

fun func(lst :: List) block:
  var result = ...  # what to return if the input list is empty
  for each(item from lst):
    # combine item with the result so far
    result := ... item ... result
  end
  result
end

There are a few new language features used in this example, introduced in the following several sections.

9.1.8.5.1 Variables that can change

First, note that we introduce the variable result with var result – this means that it can vary, which is important for the use with for each.

By default, all variables in the program directory can never be changed. i.e., if I define a variable x, I can not redefine it later:

x = 10
# ...
x = 20 # produces shadowing error

If we do want to change (or mutate) a variable in the directory later, we can, but we must declare the variable can change – as in, when we define it, rather than writing x = 10, we must write var x = 10. Then, when we want to update it, we can do so with the := operator, as is done in the template above.

var x = 10
# ... x points to 10 in directory
x := 20
# ... x now points to 20 in directory

Note that trying to use := on a variable that was not declared using var will produce an error, and variables can still only ever be declared once (whether with var x = ... or x = ...).

9.1.8.5.2 block notation

Another new language feature shown in these examples is that since Pyret functions by default expect only a single (non-definition) expression, we have to add the block annotation at the top, indicating that the body of the function is multiple expressions, with the final one being what the function evaluates to.

As another example, if we tried to write:

fun my-function():
  1
  2
end

Pyret would (rightly) error – since the function returns the last expression in its body, the 1 will be ignored – and is most likely a mistake! Perhaps the goal was to write:

fun my-function():
  1 + 2
end

However, since a for each expression exists only to modify a variable, functions that contain them will always have multiple expressions, and so we need to communicate to Pyret that this is not a mistake. Adding block before the : that begins the function (or, in general, wrapping any expressions in block: and end) communicates to Pyret that we understand that there are multiple expressions, and just want to evaluate to the last one. So, if we truly wanted to write a function as our first example, we could do that with:

fun my-function() block:
  1
  2
end

9.1.8.5.3 How for each works

A for each expression runs its body once for each element in the input list, adding an entry to the program directory for each element as it goes. It does not produce any value directly, so much instead rely on modifying variables (described above) to produce a computation.

Consider summing a list of numbers. We could write a function that does this, following our pattern, as:

fun sum-list(lst :: List) block:
  var run_total = 0
  for each(item from lst):
    run_total := item + run_total
  end
  run_total
where:
  sum-list([list: 5, 1, 7, 3]) is 16
end

On the concrete test input [list: 5, 1, 7, 3], the loop runs four times, once with item set to 5, then with item set to 1, then with item set to 7, and finally with item set to 3.

The for each construct saves us from writing the common code multiple times, and also handles the fact that the lists we are processing can be of arbitrary length (so we can’t predict how many times to write the common code). Thus, what happens is:

run_total = 0
run_total = run_total + 5
run_total = run_total + 1
run_total = run_total + 7
run_total = run_total + 3

9.1.8.5.4 Testing and variables that can change

We intentionally showed a very particular pattern of using variables that can change. While there are other uses (explored in part in Modifying Variables), a main reason to stay with this particular template is the difficulty in testing and correspondingly, understanding, code that uses them in other ways.

In particular, note that the pattern means that we never define a variables that can change outside a function, which means it can never be used by different functions, or multiple function calls. Each time the function runs, a new variable is created, it is modified in the for each loop, and then the value is returned, and the entry in the program directory is removed.

Consider what happens if we don’t follow our pattern. Let’s say we had the following problem:

Exercise

Given a list of numbers, return the prefix of the list (i.e., all elements, starting from the beginning) that sums to less than 100.

Having learned about mutable variables, but not following the pattern, you might come up with code like this:

var count = 0

fun prefix-under-100(l :: List) -> List:
  var output = [list: ]
  for each(elt from l):
    count := count + elt
    when (count < 100):
      output := output + [list: elt]
    end
  end
end

Now, this might seem reasonable – we’ve used a new construct, when, which is an if expression that has no else – this only makes sense to do inside of a for each block, where we don’t need a value as a result. It is equivalent to:

if (count < 100):
  output := output + [list: elt]
else:
  nothing
end

Where nothing is a value that is used in Pyret to indicate that there is no particular value of importance.

But what happens when we use this function?

check:
    prefix-under-100([list: 1, 2, 3]) is [list: 1, 2, 3]
    prefix-under-100([list: 20, 30, 40]) is [list: 20, 30, 40]
    prefix-under-100([list: 80, 20, 10]) is [list: 80]
end

The first two tests pass, but the last one doesn’t. Why? If we run the first one again, things are even more confusing, i.e., if instead of the above, we ran this check block:

check:
    prefix-under-100([list: 1, 2, 3]) is [list: 1, 2, 3]
    prefix-under-100([list: 20, 30, 40]) is [list: 20, 30, 40]
    prefix-under-100([list: 80, 20, 10]) is [list: 80]
    prefix-under-100([list: 1, 2, 3]) is [list: 1, 2, 3]
end

Now the test that passed at first no longer passes!

What we are seeing is that since the variable is outside the function, it is shared across different calls to the function. It is added to the program directory once, and each time we call prefix-under-100, the program directory entry is changed, but it is never reset.

Intentionally, all other uses of mutation have been on directory entries that were created only for the body of the function, which meant that when the function exited, they were removed. But now, we are always modifying the single count variable. This means that every time we call prefix-under-100, it behaves differently, because it not only do we have to understand the code in the body of the function, we have to know the current value of the count variable, which is not something we can figure out by just looking at the code!

Functions that behave like this are said to have "side effects", and they are much harder to test and much harder to understand, and as a result, much more likely to have bugs! While the above example is wrong in a relatively straightforward way, side effects can cause extremely subtle bugs that only happen when functions are called in particular orders – orders that may only arised in very specific situations, making them hard to understand or reproduce.

While there are some places where doing this is necessary, almost all code can be written without side effects, and will be much more reliable. We will explore some cases where we might want to do this in Modifying Variables.