Masters in Data Science applied to agricultural and food sciences, environment, and forestry engineering.
Instructor: Manuel Campagnolo (mlc@isa.ulisboa.pt)
code filename.py to create a new filels to list files in foldercp filename newfilename to copy a file, e.g. cp ..\hello.py farewell.py (.. represents parent folder)mv filename newfilename to rename or move file, e.g. my farewell.py goodbye.py or mv farewell.py .. (move one folder up)rm filename to delete (remove) filemkdir foldername to create new foldercd foldername change directory, e.g. cd ..rmdir foldername to delete folderclear to clear terminal windowNotes: Lecture 0 Creating Code with Python; Functions; Bugs; Improving Your First Python Program: Variables, Comments, Pseudocode; Further Improving Your First Python Program; Strings and Parameters; A small problem with quotation marks; Formatting Strings; More on Strings.
Do before next class. Video on CS50 Video Player or YouTube: follow video and recreate exercises on VS Code up to 59’ approximately (up to the section on integers ‘int’).
All topics to prepare before next class: Integers or int; Readability Wins; Float Basics; More on Floats; Def; Returning Values; Conditionals, if Statements, Control Flow, or, and, How to organize a program with main() and auxiliary functions.
Topics to prepare before next class: Modulo; Creating Our Own Parity Function; Pythonic; match, Loops; While Loops; For Loops; Improving with User Input; More About Lists; Length
Topics to prepare before next class: Dictionaries, More on code modularity (Mario example), Exceptions, Runtime Errors, try, else, Creating a Function to Get an Integer, pass
Topics to prepare before next class: Libraries, Random, Statistics, Command-Line Arguments, slice, Packages, APIs, Making Your Own Libraries
pytest: examples*argsand **kwargs. Check examples scripts in the ETC folder.Presentation of some topics from Lecture 9: args (*) and kwargs (**), list comprehensions, lambda functions, map, filter, dictionary comprehensions. Check examples scripts in the ETC folder.
Example: distinct possible ways of creating a list with all the cubes of even numbers up to N. In the last cases, map applies a given function to each element of a list. Likewise, filter applies a boolean function to filter elements of a list. Both function can be executed in parallel over the elements of the list since each output is independent of the outputs for the remainder elements of the list.
def cube(x):
return x*x*x
L=[cube(x) for x in range(N) if x%2==0]
filter to select even numbers and mapto compute cubes:
def even(x):
return x%2==0
numbers=list(range(N))
even_numbers=list(filter(even, numbers))
cubes=list(map(cube,even_numbers))
filter and map but defining implicitly the cube and even functions with lambda instead of def:
numbers=list(range(N))
even_numbers=list(filter(lambda x: x%2==0, numbers))
cubes=list(map(lambda x: x*x*x,even_numbers))
lambda and list comprehension. In the example below, if one would just write lambda x: x*x*x instead pf (lambda x: x*x*x)(x) the output list would be a list of lambda functions.
L=[(lambda x: x*x*x)(x) for x in range(N) if x%2==0]
Observations about using vectorization to speed-up computations.
A conditional can be replaced by arithmetic and logical operations. For example, let L=[x/5 for x in range(-10,10)] be a list of equally spaced numbers between -2 and 2, and say we want to apply the stepwise linear sigmoid function to the elements of the list. One could define sigma as below and then apply it to L with map.
def sigma(x):
if x>1:
return 1
elif x<-1:
return -1
else:
return x
list(map(sigma,L))
However, instead of using the conditional if once can define sigma in a simpler way with logical and arithmetic operations.
def sigma(x):
return (x>=1)*1+(-1<x<1)*x+(x<=-1)*-1
list(map(sigma,L))
Note that we don’t need to define explicitly sigma: it could be encapsulated in a lambda function lambda x: (x>=1)*1+(-1<x<1)*x+(x<=-1)*-1 as the first argument of map. One advantage of using arithmetic and logical operations is that computations can then be easily vectorized, which allows to speed them up, since the processing time for a pair of vectors is similar to the processing time for a pair of numbers.
Similarly to list comprehension, Python provides dictionary comprehension which allows to create dictionaries. The example below uses ordwhich is a function that returns the ascii index of a character, to create a dictionary of vowels, where the key is the vowel in uppercase and the value is the rank of the letter starting at 0 for a.
{x.upper(): ord(x.lower())-97 for x in 'aeiou'} # returns {'A': 0, 'E': 4, 'I': 8, 'O': 14, 'U': 20}
The main tutorial for the class is CS50P. The table below compares the contents of that course with another well-known free online Python Programming course at PP.fi. The CS50P course problem sets tend to be difficult, while you can find at PP.fi a set of problems with a larger range of difficulty, from very easy to advanced. Both online courses provide a platform for coding and testing the corretness of the solutions. Both courses provide recorded lectures: CS50P and PP.fi.
| CS50P | Contents | PP.fi | Contents |
|---|---|---|---|
| Lecture 0 | Creating Code with Python; Functions; Bugs; Strings and Parameters; Formatting Strings; More on Strings; Integers or int; Readability Wins; Float Basics; More on Floats; Def; Returning Values | Part 1 | Information from the user; More about variables; Arithmetic operations; Conditional statements |
| Lecture 1 | Conditionals, if Statements, Control FlowModulo; Creating Our Own Parity Function; Pythonic; match | Part 2 | Programming terminology; More conditionals; Combining conditions; Simple loops |
| Lecture 2 | Loops; While Loops; For Loops; Improving with User Input; More About Lists; Length; Dictionaries, More on code modularity | Part 3 | Loops with conditions; Working with strings; More loops; Defining functions |
| Part 4 | The Visual Studio Code editor, Python interpreter and built-in debugging tool; More functions; Lists; Definite iteration; Print statement formatting; More strings and lists | ||
| Part 5 | More lists; References; Dictionary; Tuple | ||
| Lecture 3 | Exceptions, Runtime Errors, try, else, Creating a Function to Get an Integer, pass | Part 6 | Reading files; Writing files; Handling errors; Local and global variables |
| Lecture 4 | Libraries, Random, Statistics, Command-Line Arguments, slice, Packages, APIs, Making Your Own Libraries | Part 7 | Modules; Randomness; Times and dates; Data processing; Creating your own modules; More Python features |
| Lecture 5 | Unit Tests; assert; pytest; Testing Strings; Organizing Tests into Folders | ||
| Lecture 6 | File I/O; open; with; CSV; Binary Files and PIL | ||
| Lecture 7 | Regular Expressions; Case Sensitivity; Cleaning Up User Input; Extracting User Input | ||
| Lecture 8 | Object-Oriented Programming; Classes; raise; Decorators; Class Methods; Static Methods; Inheritance; Inheritance and Exceptions; Operator Overloading | Part 8 | Objects and methods; Classes and objects; Defining classes; Defining methods; More examples of classes |
| Part 9 | Objects and references; Objects as attributes; Encapsulation; Scope of methods; Class attributes; More examples with classes | ||
| Part 10 | Class hierarchies; Access modifiers; Object oriented programming techniques; Developing a larger application | ||
| Lecture 9 | set; Global Variables; Constants; Type Hints; Docstrings; argparse; Unpacking; args and kwargs; map; List Comprehensions; filter; Dictionary Comprehensions; enumerate; Generators and Iterators | Part 11 | List comprehensions; More comprehensions; Recursion; More recursion examples |
| Part 12 | Functions as arguments; Generators; Functional programming; Regular expressions |
You can find information on basic concepts and features of the Python language and system at The Python Tutorial at python.org.
A nice interactive site is W3schools’ Python Tutorial where you can find in particular an easy to use Python reference documentation.
Another nice and very clear series of videos on Python are available in this Youtube channel: Python Tutorials by Corey Schafer. In particular, you might be interested in the following specific topics discussed in class:
List of both simple and more advanced programming exercises that you can run and test online: programming-23.mooc.fi. Sign-in to have access to code editor and testing.
Compiled list of CS50P proposed problems (to be completed)
Calculation of the Enteric Fermentation Methane Emission Factor of Cows
Forecast Hunter: A toolbox for assessing seasonal weather forecast data usability on ETo estimation
Pest Tracker: simular uma aplicação para a previsão da praga da mosca da azeitona no olival
Fertilização: auxiliar a fertilização com base em resultados de análises de solo