We want to clearly define some terms that we'll be using throughout the book. We're going to build up our Python understanding from this foundational terminology. In the computer world, many concepts are new, and we'll try to make them familiar. Further, some of the concepts are abstract, forcing us to borrow existing words and extend or modify their meanings. We'll also define them by example as we go forward in exploring Python.

This section is a kind of big-picture road map of computers. We'll refer back to these definitions in the sections which follow.

In Terminology we provided a kind of road map to computers. Here, we're going to look a little more closely at these things called “programs”.

What — Exactly — is the Point? The essence of a program is the following: a program sets up a computer to do a specific task. We could say that it is a program which applies a general-purpose computer to a specific problem. Sometimes we call them “application programs” because the programs are applied to definite data processing needs.

There is a kind of parallel between a computer system running programs and a television playing a particular TV show. Without the program, the computer is just a pile of inert electronics. Similarly, if there is no TV show, the television just plays noise and hisses.

We're going to focus on two parts of a program: data and processing. We'll be aiming at programs which read and write files of data, much like our ordinary desktop tools open and save files. We aren't excluding game programs or programs that control physical processes. A game's data is the control actions from the player plus the description of the game's levels and environments. The processing that a game does matches the inputs, the current state and the level to determine what happens next. An interactive game, however, is considerably more complex than a program to evaluate a file that has a list of our stocks.

Program Varietals. At this point, we need to make a distinction between some varieties of programs: specifically, a binary executable and a script. A binary executable or binary application is a program that takes direct control computer's processor. We call it binary because it uses the binary codes specific to the processor chip inside the computer. If you haven't encountered “binary” before, see Binary Codes. Most programs that you buy or download fit this description. Most of the office applications you use are binary executables (NeoOffice/J is the notable exception.) A web browser, for example, is a binary executable, as is the python program (named python.exe in Windows.)

Your operating system (for example, Windows or GNU/Linux or MacOS) is a complex collection of binary executables. Primarily, when the computer starts running, a “kernel” of binary software is loaded; this kernel controls everything that goes on in the computer. Once this kernel of software is running, it then loads a number of additional programs that constitute the working operating system with which we interact. These programs don't solve any particular problem, but they enable the computer to be used by non-engineers.

A binary executable's direct control over the processor is beneficial because it gives the best speed and uses the fewest resources. However, the cost of this control is the relative opacity of the coded instructions that control the processor chip. The processor instruction codes are focused on the electronic switching arcana of gates, flip-flops and registers. They are not focused on data processing at a human level. If you want to see how complex and confusing the processor chip can be, go to Intel or AMD's web site and download the technical specifications for one of their processors.

One subtlety that we have to acknowledge is that even the binary applications don't have complete control over the entire computer system. Recall that the computer system loads a kernel of software when it starts. All of the binary applications outside this kernel do parts of their work by using program fragments provided by the kernel. This important design feature of the operating system assures that all of the application programs share resources politely. One of the kernel's two jobs is to coordinate among the application programs. If every binary application simply grabbed resources willy-nilly, one badly behaved program could stop all other programs from working. Imagine the tedium of quitting your browser to make notes in your word processor, then quitting your word processor to go back to your web browser. The other of the kernel's two jobs is to embody the abstraction principle and make a wide variety of processors have a nearly identical set of features.

Layers of Abstraction. Let's take a close look at our metaphor again. We said there is a strong parallel between a computer running a program and a TV playing a particular TV show. We now have two layers of meaning here:

Figure 1.1. Layers of Abstraction

Our programs (and our data) reside in two places. When we're using a program, it must be stored in memory. However, memory is volatile, so when we're not using a program, it must reside on a disk somewhere. Since our disks are organized into file systems, we find these programs residing in files.

When we look at the various files on our computer, we'll see a number of broad categories.

When we use the Finder's Get Info to look at the detailed information for an application icon in MacOS or GNU/Linux, we can see that our application program icons are marked “executable” and the file type will be “application”. In Windows, a binary executable program must have a file name that ends with .exe (or .com, but this is rare).

Starting A Program. Our various operating systems give us several user interface actions that will load a program into memory so that we can start to use it. Since starting a program is the primary purpose of an operating system, there are many ways to accomplish this.

All of these actions are just different ways to get the operating system to locate the binary executable, load it into memory and give it the resources to do its unique task.

All of these choices boil down to two overlapping paths to a starting a binary executable:

In the section called “What is a Program?” we looked at what a program is. Here we'll look at what it means when the operating system runs a program. After this, we can revisit Python, and define what a script program is in the section called “The Python Program and What It Does”.

Computer use is a goal-directed activity. When we're done using our software, we've finished some task and (hopefully) are happy and successful. We can call this a state change. Before we ran our program, we were in one mental state: we were curious about something or needed to get some data processing done. After we ran our program, we were in a different mental state of happy and successful. For example, we got the total value of our stock portfolio from a file of stock purchases.

We like this “state of being” view of how programs work. We can think of many things as state changes. When we clean our office, it goes from a state of messy to a state of clean (or, in my case, less messy). When we order breakfast at the coffee shop we go through a number of state changes: from hungry to waiting for our toast, to eating our toast, to full and ready to start the day.

Let's work backwards to see what had to happen to get us to a happy and successful state of being.

Success! The program has done the desired job, cleaned up, and we are back looking at the operating system interface (the Finder, Explorer or a Terminal prompt). We say that the program has reached a terminating state. Therefore, one goal of programming is to create a program that finishes it's work normally so that the operating system can deallocate the resources and regain control of our computer.

Running. The program is running, doing the job we designed it to do. Perhaps it is controlling a device or computing something. The program is undergoing a number of internal state changes as it moves from its initial or start-up state to its terminating state. Often it reaches terminating state because we clicked the Quit menu item. Another goal of programming, then, is to have the program behave correctly when it is running.

Starting. The operating system loads the program into memory from files on a disk. The operating system may load additional standard modules that the program requires. The operating system also creates a schedule for our program. Most operating systems interleave several activities, and our program is only one of many programs sharing the time and memory of the computer. Once everything is in place we see it start running. Another goal of programming is to have a program that cooperates with the operating system to start in a simple way and follow all the rules for allocating resources.

We'll think about programming from a number of viewpoints. This sequence of activities is one point of view: we see a program go through start-up, processing and termination. This isn't the only viewpoint, we'll look at several others. Sometimes we call this the dynamic view of our program because it reflects the various changes in state.

In the section called “What is a Program?” we noted the difference between a binary executable and a script. We defined a binary program as using codes that are specific to our processor chip and operating system. A script, on the other hand, is written in an easy-to-read language like the Python language. It turns out that they are intimately intertwined because the scripts depend on a binary executable.

The Python program, python or python.exe, is described as an interpreter. The Python program's job is to read statements in the Python language and execute those statements. For historical reasons, this process is called “interpreting” the program. You might think that an interpreter should be translating the program into another language; after all, that's what human interpreters do. Software people have bent the meaning of this term, and the process of executing a script is called interpreting.

Ultimately, everything that happens in a computer is the result of the processor executing it's internal binary instructions. In the case of Python, the python program contains a set of binary instructions that will read the Python-language statements we provide and execute those statements.

Looking Under the Hood. This is the real story behind the two kinds of executable programs we run on our computers.

At this point our TV playing a TV show metaphor (from Chapter 1, About Computers) is starting to look a little shabby. Originally, we said there is a parallel between a computer running a program and a TV playing a particular TV show. With computers, however, we have three layers of meaning.

Software builds up in stacks and layers, based on the principle of abstraction. TV simply switches channels. It looks like computers are so strange that metaphors will only cause more confusion. There aren't many things that are like computer systems where the behavior we see is built up from independent pieces. We can't really talk about them metaphorically, which makes computers a unique intellectual challenge.

Here's a picture that shows the abstract Computer-Plus-Operating system. In one case, it is controlled by a binary executable. In other case, it is controlled by Python, and our Python-language program controls Python.

Figure 2.1. The Python Abstract Computer

Bottom Line. When we write Python-language statements, those statements will control the Python program; the Python program controls the OS kernel; the OS kernel controls our computer. These are the most obvious and influential layers of abstraction. It turns out that there are other parts of this technology stack, but we can safely ignore them. The OS makes hardware differences largely invisible; and Python makes many OS differences invisible.

The cost of these layers of indirection is programs that are somewhat slower than those which use the computer's internal codes. The benefit is a huge simplification in how we write and use software: we're freed from having to understand the electro-techno-mumbo-jumbo of our processor chip and can describe our data and processing clearly and succinctly.

Running our Python-language Programs. Remember that the Python program's job is to read statements in the Python language and execute those statements. Our job as a programmer is to write the statements that tell the Python program what to do.

From the operating system's point of view, all of our various Python programs and tools are really just the python program. When you double click the file you created (usually a file with a name that ends in .py) this is what happens under the hood. Let's pretend you're running a program you wrote named roulette.py.

While the coffee-shop answer is “programming is how we create programs”, that doesn't help very much. We can identify a number of skills that are part of the broadly-defined craft of programming. We'll stick to two that are foundational: designing Python statements and debugging problems with those statements. Design comes in a number of levels of detail, and we'll work from the smallest level up to larger and more inclusive levels

We take much of our guidance on this from Software Project Management [Royce98]. Royce identifies four stages of software development, with distinct kinds of activities and skills.

We're going to focus on two skills in this book: creating Python language statements, and debugging problems when we make mistakes. Testing is a rich subject; it would double the size of this book to talk about appropriate testing techniques. The analytical skills for inception and elaboration don't require knowledge of Python, just common sense and clear thinking.

This is a useless digression. It may help you understand how the team that wrote the Python program did it. It can help you demystify programming. It may not help you learn the Python language, so feel free to skip it.

Binary executable files are created by a program called a compiler. A compiler translates statements from some starting language into the processor's native instruction codes. This leads to blazing speed. This approach is typified by the C language. One consequence of this is that we must recompile our C language programs for each different chip set and operating system.

The C language isn't terribly easy to read. The language was designed to be relatively easy for the compiler to read and translate. It reflects an older generation of smaller, slower computers.

The GNU Tools. For the most part, the GNU C Compiler and C language libraries are used to write binary executables like Python. The C language has been around for decades, and has evolved a widely-used style that makes it appropriate for a variety of operating systems and processors. The GNU C compiler has been designed so that it can be tailored for all processors currently used to build computers. Many companies make processors, include Intel, National Semiconductor, IBM, Sun Microsystems, Hewlett-Packard, and AMD. The GNU C Compiler can produce appropriate binary codes for all of these various processor chips.

In addition to the processor (or “chip architecture”), binary executables must also be specific to an operating system. Different operating systems provide different kernel services and use different formats for their binary executable files. Again, the GNU C Compiler can be made to work with a wide variety of operating systems, producing binary executable files with all the unique features for that operating system.

The ubiquity of the GNU C compiler leads to the ubiquity of Python. By depending on the GNU C compiler, the authors of Python assured that the python program can be compiled for any processor chip and any operating system.

There's a distinction between download and install that sometimes escapes newbies.

When you go to www.python.org and download some software, you wind up with a file on your computer. But this file isn't in a location where the operating system can find it.

Generally, software posted on the internet is compressed into a small file. On your computer, it generally exists as a number of files, often in an expanded form which is easier to process, but larger. When you install a piece of software, the installer both uncompresses the files and distributes them to their proper locations.

The installaer can take a number of forms, dependong in your operating system.

The casino table games of Craps and Roulette (and a number of similar games) allow the bettor to place a bet (or wager) on an outcome or set of outcomes. Some random device (cards, dice, a wheel, a spinner) is used to make a selection. This selection usually resolves the bets as winners, losers, or a “push” where your money is returned.

In Roulette, each random event defines a complete set of outcomes, and all bets are resolved. You see this in the play at the Roulette table: people place bets, the wheel is spun, all of the bets are resolved. Once the bets are resolved as winners or losers, players are permitted to bet again.

In Craps, each random event does not define a complete set of outcomes. Some bets are not resolved when the dice are thrown; instead, the bets remain. Craps is played as a series dice throws that are part of a round or turn. The turn can be as short as a single throw of the dice, or it can be indefinitely long. It is unlikely (about a 1% chance) for a turn to take more than seven throws of the dice, but not impossible.

Generally, the person throwing the dice, the “shooter”, holds the dice as long as they win their round. When they lose, the dice move to a new shooter. These nuances of casino play has no impact on the actual game, so we'll ignore details like these.

Odds. If the outcome you bet on is likely, your payout is rather small. If the outcome you bet on is rare, your payout may be huge. They call this the odds of winning. When the odds are small, the event is pretty likely. For example, almost half the Roulette wheel has numbers colored red. Betting on red, then, is pretty safe. Since it's about half the numbers, the payout is 1:1. If you bet $10, you could win an additional $10.

Contrast red (or black) with the number zero, which is just one of the thirty eight bins on the wheel. Since zero is so rare, it pays off at 35:1. If you bet $10 on zero, and it comes up, you could win $350. They call these long odds or a long shot.

Here's the short form of the question: “How well does the Martingale betting system work for Roulette?”. After we define any unknown jargon in this question, we'll see that it is not terribly complex and it will lead us to some related questions. All of these questions can be answered with simple Python programs.

Roulette. In Roulette players make bets and wheel is spun to determine which bets win and which bets lose. The Roulette table has a number of positions on which you can place bets by stacking up chips. The Roulette wheel is a collection of numbered bins. When the wheel is spun, a small ball is dropped into it, and the ball will eventually come to rest in one of the bins. The bin selected by the ball determines which of the betting positions are winners and which are losers. Each position has a payout ratio that determines how much you win based on how much you bet.

There are over a hundred possible bets on the Roulette table, and a wide variety of payout ratios. We'll define a few of them, and focus on just six of the available bets.

Martingale Betting. The Martingale betting system suggests that you organize your casino play as follows:

Now, let's look at our question again. How well does this Martingale system work? We can see that the green zero and double zero complicate the analysis. There are ways to work out the details, but rather than learn a lot of math, we'll learn a little Python and simulate the whole thing. We can collect some statistics showing the results of our simulated Roulette game.

We can ask a whole family of related questions by replacing the Martingale betting system with more complex systems. We can ask questions based on extending the Martingale system to include additional bets. This is the beauty of writing our own simulation: we can modify our program to try out different variations on our betting procedure.

Here's the short form of the question: “How well does the Field bet pay in Craps?”. We'll define the gambling jargon and then look at this question again, in a little more detail.

In Craps, players make bets and a pair of dice are thrown to determine the state of the game. Some dice throws are significant events and will resolve some or all of the bests as winners or losers. Some dice throws are less significant and resolve some bets. Some dice throws don't change the state of the game at all.

The Craps table has a number of positions on which you can place bets by stacking up chips, as well as a token that shows the state of the game. A shooter will throw two dice; the number on the dice will do several things. First, the number will pay off any proposition bets based on just this throw of the dice. Second, the number will pay off any of the various number bets that can be placed. Third, the number may change the state of the game, which can also resolve certain kinds of bets.

The Craps game has two states: point “off” and point “on”. The casino will place a large black and white disk on the table to show the state of the game.

Point Off or the Come Out Roll. The first time the shooter throws the dice, the point is off. If the shooter throws 7 or 11, this turn is an immediate winner, and bets are resolved. If the shooter throws 2, 3 or 12, this turn is an immediate loser and bets are resolved. In this case, the point is still off, the game didn't change state, it's still just beginning.

When the point is off, and the shooter throws 4, 5, 6, 8, 9 or 10, the game changes state, and now the point is on. The casino will flip over the large disk to show the "on" side, and put it on the betting space that shows the point number.

Point On. When the point is on, a number of additional bets are allowed. Additionally, the disk sits in a number's space to prevent certain other bets. We'll avoid the complexity of these conditionally permitted bets. In a casino, however, you would see a flurry of activity when a point is established.

When the point is on, and the shooter throws this point number, the round is a winner, and most of the bets are resolved. There are some bets that will persist, however. When the shooters throws a seven, the round is a loser, and all bets are resolved.

Throwing a seven means the shooter lost, and most bets are losers. There are, however, some “don't” bets that will be winners when the shooter doesn't win. These are sometimes called “wrong bets”, and involve a more sophisticated odds calculation. In general, you can put up a lot to win a little when you make wrong bets.

When the shooter throws 2, 3, 11 or 12, nothing much can happen. Certain one-roll proposition bets are resolved, but these four numbers are neither points nor are they 7, which ends the game.

Other Bets. There are a number of bets which don't depend on the state of the game. These are one roll “proposition” bets. The field is one of these bets. You place your bet in the box marked “Field” before the dice are thrown. The number on the dice determines the field bet result immediately.

The field bet wins on any of 2,3,4,9,10,11, or 12. The 3,4,9,10, and 11 pay 1:1 (“even money”) and the 2 and 12 pay off at 2:1.

Analysis. There are a number of questions about the field bet. We can create a simple simulator to see the basic outcome. We can use a more sophisticated simulation of doing Martingale betting (see the section called “The Roulette Problem”) to see how that changes the performance of this bet. Some people use an even more complex betting system for the field by increasing their bet with each win and decreasing it with each loss. We'll stick with a simple simulation as a way to learn Python

We aren't going to describe the solutions to any of these casino game problems here — that would rob you of the intellectual fun of working out your own solutions to these problems. Instead, we want to provide some hints and nudges that will parallel the course this book will take.

This may already be obvious, but we're going to address these problems by writing new software in the Python language. The reason why it is important to restate the (potentially) obvious is that in Using Python we're going to spend time on learning to control the python program in a simple, manual way. Then, when we write programs, we'll control python with our programs to do more sophisticated work.

Any solution to these kinds of problems will involve some simple math. Almost all computing involves some kind of math. Business programming tends to involve the simplest math. Engineering and science can involve some really complex math. Statistics is often in the middle ground, which is why we will look at it closely in Arithmetic and Expressions.

By the way, in addition to math-oriented computing, there is also computing that could be termed “symbolic” in nature. It might involves words or XML documents or things that aren't obviously mathematical; we'll set this aside as atypical for newbies.

Sequential Thinking. A program in Python is often a sequence of operations. In the casino game definitions, we saw that each game was a sequence of individual steps. We can often summarize programs by looking at their inputs, their processing steps and their outputs. This input-process-output model reflects the sequential order of processing: first, read the inputs; second, do the processing; third, print the outputs. More sophisticated programs (like games or web servers) will interleave these operations. We'll look at this in Programming Essentials.

The sequence of operations is rarely fixed and immutable. With casino games, we have some bets which are winners and some bets which are losers. We have conditional operations of collecting losing bets and paying winning bets. Additionally, we'll have some operations which have to be repeated for a number of simulations, or until some condition is satisfied. We'll look at this in Some Self-Control.

Our exploration of Python starts with arithmetic expressions and moves on to statements, then to sequences of statements. We'll add conditional and iterative statements. The next step will be a simple organizing principle called a function definition. We'll introduce this in Organizing Programs with Function Definitions and use it to package parts of our program until a useful, discrete components that can help us control the overall complexity of our program.

Other Side Of the Coin. Beginning with Getting Our Bearings we'll turn to a different tack. The first parts of our exploration were focused on the processing, and the procedural nature of our problems. The second part of our exploration will look at the data and collections of data.

If we are going to simulate a number of sessions at the Roulette wheel, following our Martingale strategy, we'll need to collect the results and do statistical analysis on the collection. We'll look at collections of data items in Basic Collections of Data: Strings, Lists and Tuples.

We'll address some programming techniques in Additional Processing Patterns — Exceptions and Iterations that make our Python programs more reliable and also a bit simpler. Simplification is a touchy subject: simplifications aren't always appreciated until you see the more complex alternative. Further, since we're approaching Python by moving from the elementary to the advanced, some things we'll look at will be complex but elementary. As we learn more, we can replace them with something simple but advanced.

In More Data Collections: Sets, Mappings, Dictionaries and Files we'll look at some additional data structures that can help us develop truly useful solutions to our problems. These additional data structures will give us foundational knowledge of the Python language and the built-in data types that we can use.

Successful Collaboration. When we look at our problems, we see that there is considerable interaction among a number of objects. For example, in Roulette, we have the following kinds of things:

This interaction between player, table and wheel forms a larger thing, called the game, which lasts until the player wins big, loses big, or has spent too much time at the table. Each game produces a final result of zero dollars, big bucks or some number of dollars that was available when time ran out. These, in turn are collected for statistical analysis. An even bigger assembly of objects does the simulation and analysis. We'll learn how to define these collaborating objects in Data + Processing = Objects.

A lot of the basic components that make a program robust and reliable are already packaged as Python modules, and we'll cover these in Organizing Programs with Modules. We'll also use the built-in modules as templates for designing our own modules; this allows us to organize our program neatly into discrete, easy-to-manage pieces.

Our final section, Fit and Finish: Complete Programs, will cover some final issues. These are the things that separate a fragile mess that almost works most of the time from a useful program that can be trusted.

Python reflects a number of growing trends in how people develop new computer programs. It is a very simple language, supported by an interpreter and surrounded by a vast library of add-on modules. It is an open source project, supported by dozens of individuals; this encourages you to build complete solutions from smaller components and partial solutions. We'll look at each of these facets separately.

Planet Python. Python is really four separate elements in a single, tidy package. I like to think of it as an wonderfully efficient planet that we can visit. To get things done on that planet you have to learn the language. Once you've learned the language, however, you find that the whole planet is organized to do everything you ask precisely and very quickly. Like any well-run organization, it has a number of services that make life convenient and safe, and assure the common good of all the inhabitants. Finally, it offers a kind of public forum for making your requests and seeing the results of those requests.

This mythical Planet Python is the Python program itself, we'll call it python in this book. Windows users will know it as python.exe. The Python program, python, runs on your computer, and carries out statements written in the Python language. The program has just one purpose — execute Python language statements — so it is small and efficient. Because it is so tightly focused, it is wonderfully reliable.

The planet's services are the Python libraries. These libraries include programs you can extend, and pieces of programs that you might use to create a more complete program. Some parts of the libraries are both: things you extend to add new features, and then use in your final program. I think of these as essential services like police departments, public libraries, laundromats, and telephone sanitizers. You build your complete organization or enterprise using these pre-built organization units.

The public forum is the integrated development environment (IDE). This is the environment where you develop your Python program. It's integrated because all the tools you might want are right there in a single program. In this case, the program's name is IDLE. You use IDLE to write Python statements, execute sequences of Python statements and read any resulting messages.

Simplicity. Python is a relatively simple programming language that allows you to express data processing in clear, precise terms. The Python language has an easy-to-use syntax, focused on the programmer who must type, read and understand a program. The language is designed to look a bit like a natural language, with simple punctuation and indentation. Computer languages are more rigid than human languages: when you misspell something in English, people can often determine what you meant, and make sense of what you wrote. The Python program, python, is only a simple piece of software: spelling and punctuation really matter. While Python is easier than most other programming languages, you must still be precise.

Interpreted. The computer science folks characterize the Python program, python, as an interpreter: it interprets and executes the Python language statements, doing your data processing. Because it is interpreting our statements, it can provide useful diagnostic information when something goes wrong. It can also handle all of the tedious housekeeping that is part of how programs make use of the computer's resources. As users, we don't see this housekeeping going on, and as newbie programmers we shouldn't have to cope with it, either.

The computer-science types make a distinction between interpreters (like Python) and compilers (used for the C language). The C compiler (controlled by a program named cc) translates C language statements into a program that uses the hardware-specific internal codes used by your computer. The operating system can then directly execute that resulting program. After you see the results of execution you might make changes, recompile and re-execute. This compilation step makes everything you do somewhat indirect. The compiler translates your C statements into another language which is then executed. This indirection makes compiled languages harder to learn; it also makes diagnosing a problem very hard.

Here's a diagram that may help clarify how Python differs from a language like C. For a C programmer, they will use a complex IDE which includes the C Compiler to translate their C statements into a binary executable program from their statements. For a Python programmer, a simpler IDE uses the python program to execute the Python statements.

Figure 5.1. C Compiler vs. Python

The binary executables have relatively direct control over the operating system and computer. A Python-language program controls Python.

Libraries. Python, the project, includes a rich set of supporting libraries. These libraries contain the basic gears, sprockets, flywheels and drive-shafts that you can use to make a program. By separating the library tool-boxes from the core language, the designers of Python could keep the language simple, which means the interpreter can be very efficient and reliable. Yet, they can provide an extensive feature set as separate extensions. Every new idea can be added as another extension.

There are other consequences to having extensive and separate libraries. Principally, good ideas can be preserved and extended, and bad ideas can be ignored. This basic evolution saves programmers from having to design everything perfectly the first time. As you get more experience with the Python programming community, you will see ideas come and go. Some extensions will blossom and become widely used, where others will be quietly ignored because something better has come along.

Another consequence of having separate libraries is that any programming project should begin with a survey of available libraries. This can replace unproductive programming with more productive research and reuse.

Development Environment. Finally, we see that Python also comes with a development environment, or workbench, that you can use to write and execute your Python statements. The integrated development environment (IDE) includes an editor for writing Python files, and the Python interpreter, plus some other tools for searching the Python libraries.

Interestingly, the Python development environment is just another Python program. When you double-click on the IDLE icon, you are starting a Python program that helps you write Python programs. At first, this seems like a real mind-wrenching problem. You might think of it as similar to asking “which came first, the chicken or the egg?”. It isn't all that bad a problem however. In this case, someone else wrote IDLE to help you write your program. Your program, and IDLE (and a large number of other programs) all share the Python program as the driving engine.

Timeline. The Python programming language was created in 1991 by Guido van Rossum based on lessons learned doing language and operating system support. Python is built from concepts in the ABC language and Modula-3. For information ABC, see The ABC Programmer's Handbook [Geurts91], as well as http://www.cwi.nl/~steven/abc/. For information on Modula-3, see Modula-3 [Harbison92], as well as http://www.research.compaq.com/SRC/modula-3/html/home.html.

The current Python development is centralized in SourceForge. See http://python.sourceforge.net for the latest developments.

The command prompt is sometimes hard to find in Windows. In Windows 2000, you have to look in the Start menu under Programs, and then under Accessories to find the Command Prompt.

You can also use the Run... menu item in the Start menu. This will give you a small dialog box where you can type the name of a program. The name of the command prompt is just cmd. You can type cmd, and click Okay.

When you run the command tool, it will present a black window with a prompt from the operating system that looks something like C:\Documents and Settings\SLott>. Here, you can type the word python, hit return, and you're off and running.

In the Applications folder, you'll find a Utilities folder. In the Utilities folder, you'll find a program named Terminal. Double click Terminal and you'll get a window with a prompt from the operating system that looks something like [DVDi-Mac-1:~] slott%. Here, you can type env python, and you're off and running.

You might want to drag the Terminal icon onto your dock to make it easier to find.

The Fedora Linux desktop, for example, has a Start Here icon which displays the applications that are configured into you GNOME environment. The System Tools icon includes the Terminal application. Double click Terminal and you'll get a window which prompts you by showing something like [slott@linux01 slott]$.In response to this prompt, you can type env python2.4, and you're off and running.

For non-Gnome Linux and Unix variants, you must find your Terminal tool, into which can type python.

Once the Python program has started, it looks something like the following. It doesn't matter whether Python starts up from the command prompt or terminal window, the basic operation is the same. The window title and background color may be different, but our interaction with Python will be the same.

Microsoft Windows 2000 [Version 5.00.2195]
(C) Copyright 1985-2000 Microsoft Corp.

C:\Documents and Settings\SLott>python
Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

When we get the >>> prompt, the Python interpreter is listening to us. We can type any Python statements we want. Each complete statement is executed when it is entered.

To finish using Python, we enter the end-of-file character. We do this when we're completely done with all of our work. We don't absolutely have to do this, because we can always just exit the Command Prompt or Terminal. When we stop the Terminal, the running python will be killed.

MacOS and GNU/Linux. The polite way to tell Python that we're done is Ctrl-D.

Windows. The polite way to finish a conversation with Python is Ctrl-Z, followed by Enter.

The Python program's job is very simple: it prompts you for a statement, executes the statement you entered, and then responds back to you with the result of executing the statement. This little three-step loop is all that Python does. Of course, as we will see, the real power comes from the wide variety of statements we provide in the Python language.

Since we are all newbies to programming, we'll start with some very simple Python interactions, just to see what kinds of things Python can do. We'll start the Python program and then type Python statements that evaluate a simple formula. For these first few examples, we'll include the reminder to start running Python.

This first example will show the mathematical operation of ÷. If you look at your computer keyboard, you won't find the ÷ key. Python uses / for division.

Example 6.1. Our First Dialog: Miles per Gallon

C:\Documents and Settings\SLott>python  
Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 351 / 18  
19  

	First, I typed python to start the python program running. If this was my Fedora GNU/Linux machine, I would have used python2.4, instead.
	I typed 351 / 18 to compute miles per gallon I got driving to Newark and back home. This is a complete Python statement, and Python will evaluate that statement.
	Python responded with 19: a rotten 19 miles per gallon. I've got to get a new car that uses less gasoline.

This shows Python doing simple integer arithmetic. There were no fractions or decimal places involved. When I entered 355 / 18 and then hit Return, the Python interpreter evaluated this statement. Since the statement was an expression, Python printed the results.

The usual assumption for numbers is that they are integers, with a range of -2,147,483,648 to 2,147,483,647. This seems like a peculiar set of numbers, but it happens to be all the numbers that the creatures of planet Binome (see Binary Codes) can represent when 32 of them participate. This is what the computer scientists call a 32-bit integer. You can then ask, “why 32 bits?” This is the subject for an FAQ.

Note that Python does not like ,'s in numbers. Outside Python, we write large numbers with ,'s to break the numbers up for easy reading. Except for years on the calendar, where we omit the ,'s; we write 2007, not 2,007. Python can't cope with ,'s in the middle of numbers. The mileage on my odometer reads 19,241. But, in Python we write this as 19241.

Bottom Line. For now, be comfortable that Python is perfectly happy with whole numbers that are plus or minus two billion. Remember to avoid commas. We sometimes call these numbers ints, short for integers. Later, we'll see that Python has a pretty expansive set of numbers available to work with.

That calculation was nice, but you'll notice that whole numbers aren't really very accurate. If you pull out a calculator, you'll see that Python got a different answer than your calculator shows.

If you include a period in your numbers, you get “floating decimal point” numbers. We call these floating-point or floats. The number of digits on either side of the decimal point can “float”. Floating point numbers are handy for many kinds of calculations.

Our previous conversation used whole numbers. Let's try again, using floating-point numbers.

Example 6.2. Using Floating Decimal Point Numbers

>>> 351. / 18.
19.5

	I typed 351. / 18. to compute miles per gallon I got driving to Newark and back home.
	Python responded with 19.5: the more accurate 19.5 miles per gallon.

There's another kind of number that we'll get to later. When we do financial calculations on US dollars, the decimal point is fixed; we have two digits to the right of the decimal point and no more. These fixed-decimal point numbers aren't a built-in feature of Python, but there are ways to extend Python with a library that gives us this capability.

Bottom Line. For now, be comfortable that Python is perfectly happy with floating-point numbers that have about 17 total digits of accuracy, but a range that is huge. Remember to include a decimal point to tell Python that you want to see decimal places in the calculation. Also, remember to avoid commas, they're just confusing.

So far, we've given arithmetic expressions to Python and Python's response is to evaluate those expressions. When we look at our keyboard, we can see that we have / (for division), + for addition and - for subtraction. What about multiplication (×)? Square roots (√)? Raising to a power? These aren't keys on a standard keyboard.

Here are the arithmetic operations that Python recognizes in forms very similar to the way mathematics is written:

Additionally, Python (and many other programming languages) provide two handy operators that mathematicians don't normally write down in this form. Mathematicians may talk about “modular” arithmetic, with something like a (mod m). This is written in Python using the % character. For non-mathematicians, this is the remainder after division.

>>> 355 % 113
16

Here's what this shows us: 113 goes into 355 three times with 16 left over. Mathematically, 355 = 3 × 113 + 16.

The other mathematical operator that Python gives us is the // sequence of characters. This gives us the rounded-down — no fractions, no decimal places — result of division. At first, that seems a little silly, since 355/113 is 3. So is 355//113.

Consider 355.//113.. Above, we said that using the decimal points made this a floating decimal point calculation. Yet, the answer doesn't seem to reflect this.

>>> 355.//113.
3.0
>>> 355./113.
3.1415929203539825

Aha! When we use //, the floating-point numbers are treated like they were whole numbers, and integer-like division is done. When we use /, we get the “natural” division appropriate to the two numbers: with floating-point numbers, we get precise answers; with integers, we get an integer answer.

The usual mathematical rules of operator precedence apply to Python expressions: multiplies and divides will be done before adds and subtracts. Plus, we get to use ()'s are used to group terms against precedence rules. Unlike mathematics, we can't use []'s and {}'s in arithmetic expressions. Mathematicians can use these, but in Python, we have to limit ourselves to just ()'s.

For example, converting 65° Fahrenheit to Celsius is done as follows.

>>> (65 - 32) * 5 / 9
18
>>> ((65 - 32) * (5 / 9))
18

We have to put the 65-32 in parenthesis so that it is done before the multiply and divide.

What would happen if we said 65-32*5/9? Try it first, to see what happens.

If we don't include the ()'s for grouping, then Python would do what every mathematician would do: compute 32*5/9 first and then the difference between that and 65. Python did what we said, but not what we meant. We know the answer is wrong because 65° Fahrenheit can't be the impossibly hot 48° Celsius.

In the second example, we put in extra ()'s that don't change the resulting answer.

Python prompts us with the basic "I'm listening" prompt of >>>. When we type an expression statement, Python prints the result for us, and then another prompt.

Python has a second prompt that you will see from time to time. It indicates that your statement isn't complete, and more is required. It's the "I'm still listening" prompt of .... Here's how it works.

For this section only, we'll emphasize the usually invisible Return ( ) key. When we start using compound statements in Chapter 20, Conditional Processing: Only When Necessary, we'll add some additional syntax rules. For now, however, we have to emphasize that statements can't be indented; they must begin without any leading spaces or tabs. Here's a simplest case: converting 65° Fahrenheit to Celsius.

>>> (65 - 32) * 5 / 9 
18
>>>

What happens when the expression is obviously incomplete?

>>> ( 65 - 32 ) * 5 / 

  File "<stdin>", line 1
    (65-32)*5 /
              ^
SyntaxError: invalid syntax

>>>

This leads us to the first of many syntax rules. We'll present them in order of relevance to what we're doing. That means that we're going to skip over some syntax rules that don't apply to our situation.

Just to be complete, we'll present syntax rule two, but it doesn't really have much impact on what we're going to be doing.

There is an escape clause that applies to rule one (“one statement one line.”) When the parenthesis are incomplete, Python will allow the statement to run on to multiple lines.

>>> ( 65 - 32 
... ) * 5 / 9
18
>>>

Yes, we skipped rules three, four and five.

It is also possible to continue a long statement using a \ just before hitting Return at the end of the line.

>>> 5 + 6 *\
... 7
47
>>>

This is called an escape and it allows you to break up an extremely long statement. It creates an escape from the usual meaning of the standard meaning of the end-of-line character; the end-of-line is demoted to just another whitespace character, and loses it's meaning of “end-of-statement, commence execution now”.

We've been ignoring spaces in our expressions. There are some spaces in the examples, but we haven't been dwelling on precisely how many spaces and where the spaces are allowed. It turns out that spaces other than indentation are very flexible. Indentation is not flexible.

We have found some kinds of mistakes that we can make with unclosed ()'s and an extra \ at the end of the line. This leads us to an important debugging tip.

Simple Commands. Enter the following one-line commands to Python:

Simple Expressions. Enter one-line commands to Python to compute the following:

Python programming is often done using an Integrated Development Environment (IDE) named IDLE. The IDLE program does a number of things that make programming easier.

IDLE isn't your only choice for an IDE. It is, however, free and so easy to use that we'll focus on it in this book. There are some alternatives that might want to explore.

After we get past the operating-specific details, we'll see that the Python Shell window of IDLE is the same on all of out various operating systems.

env python /usr/lib/python2.4/idlelib/idle.py &

After we get past the operating-specific details, we see that the Python Shell window of IDLE is the same on all of the various operating systems. The Python Shell window will have the following greeting.

Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.

    ****************************************************************
    Personal firewall software may warn about the connection IDLE
    makes to its subprocess using this computer's internal loopback
    interface.  This connection is not visible on any external
    interface and no data is sent to or received from the Internet.
    ****************************************************************
    
IDLE 1.1.4      
>>> 

If you have personal firewall software and it does warn you about IDLE, you can ignore your personal firewall's messages. Your firewall is detecting ordinary activity called “interprocess communication” among the various components of IDLE. Rather than a personal firewall, I buy routers that do this for all the computers in my home.

You can use the File->Exit to exit from IDLE. You can also close the window by clicking on the close icon.

When IDLE is running, the Python Shell shows the Python >>> prompt. This is the same "I'm Listening" prompt we saw in the section called “Your First Conversation in Python: miles per gallon”. Interacting with Python through IDLE is the same as interacting with Python directly.

>>> -20 * 9/5 + 32
-4

Interesting. When the Stockholm weather says -20 Celsius, that is -4 Fahrenheit. That's cold.

Drat! We used numbers without any decimal points. That means we used integer division, which won't be very accurate. We'd like to try that statement again without having to retype the entire thing from scratch.

IDLE has a couple of features to make it possible to work more efficiently.

First, we have ordinary copy and paste capabilities. If you look on the Edit menu, you'll see the usual culprits: Cut, Copy and Paste. If you are new to this sort of thing, here's the play-by-play.

Once you have the almost-correct statement, you can use the left-arrow key to move across the line and add decimal points to make the line look like this: -20 * 9./5. + 32.

The cooler technique is to use the up-arrow key to go back to our original command. Here's the play-by-play.

Once you have the almost-correct statement, you can now use the left-arrow key to move across the line and add decimal points. This up-arrow and left-arrow editing is perhaps the easiest way to re-enter a command with minor changes.

Plain integers in Python are written as strings of digits without commas, periods, or dollar signs. A negative number begins with a single -. Plain integers have range ±2 billion, or about 9 decimal digits.

Internally, an integer is compact, using just four bytes of memory. It's also blazingly fast for most mathematical operations. However, this small size and high speed also mean that it has a limited range of values.

Here are some examples of integers. Note the absence of ,'s, .'s, $'s or other punctuation. We can only use - to mean a negative number.

Here are the basic arithmetic operations that Python recognizes:

Here are some examples.

>>> 32 - 42
-10
>>> 42 * 19 + 21 / 6
801
>>> 2**10
1024
>>> 241 % 16
1
>>> (18-32)*5/9
-8

Pay close attention to 42 * 19 + 21 / 6. In particular, remember that your desktop calculator may say that 21 ÷ 6 is 3.5. However, since these are all integer values, Python uses integer division, discarding fractions and remainders. 21/6 is precisely 3.

Does Python Round? Try this to see if Python rounds. If Python does not round, the answers will all be 2. If Python does round, the answers will be 2, 2, 3 and 3.

8 / 4
9 / 4
10 / 4
11 / 4

What happened? It shouldn't be any surprise that integer arithmetic is done very simply. For more sophistication, we'll have to use floating-point numbers and complex numbers, which we'll look at in later sections.

New Syntax: Functions. More sophisticated math is separated into the math module, which we will look at in the section called “The math Module — Trig and Logs”. Before we get to those functions, we'll look at a few less-sophisticated functions.

The absolute value (sometimes called the magnitude or absolute magnitude) operation is done using a slightly different syntax than the conventional mathematical operators like + and - that we saw above. A mathematician would write |n|, but this can be cumbersome for computers. Instead of copying the mathematical notation, Python uses a kind of syntax that we call prefix notation. In this case, the operation is a prefix to the operands.

Here's the formal Python definition for the absolute value function.

This tells us that abs(x) has one parameter that must be a numeric value and it returns a numeric value. It won't work with strings or sequences or any of the other Python data types we'll cover in Part VIII, “Basic Collections of Data: Strings, Lists and Tuples”.

Here are some examples using the abs(x) function.

>>> abs(-18)
18
>>> abs(6*7)
42
>>> abs(10-28/2)
4

The expression inside the parenthesis is evaluated first. In the last example, the evaluation of 10-28/2 is -4. Then the abs(x) function is applied to -4. The evaluation of abs(-4) is 4.

Floating decimal point numbers are written as strings of digits with one period to show the decimal point. They can't have commas or dollar signs. A negative number begins with a single -. We call them floats or floating point numbers for short.

These numbers are different from the “fixed-point” decimal numbers that we use for financial calculations. With fixed-point numbers, the number of positions to the right of the decimal place is fixed. Doing fixed-point processing in Python is done with an add-on library; we'll cover this in the section called “One Way To Tackle Fixed Point Math”.

A floating-point number takes at least eight bytes, making it twice the size of a plain integer. The extra complexity of scientific notation makes them much slower than plain integers. They have about 17 digits of useful precision, but they can represent values with an astronomical range.

Here are some examples:

We can, if we want, write our numbers in scientific notation. A scientist might write 6.022×10²³. In Python, they use the letter E or e instead of ×10. Here are some examples.

All of the arithmetic operators we saw in the section called “Plain Integers, Also Known As Whole Numbers” also apply to floating-point numbers. Here are a couple of examples.

>>> 6.62e-34 * 2.99e8
1.97938e-025
>>> 3.1415926 * 3.5**2
38.484509350000003

You Call That Accurate? What is going on with that last example? What is that “0000003” hanging off the end of the answer?

That tiny, tiny error amount is the difference between the decimal (base 10) display and the binary (base 2) internal representation of the numbers. That tiny, annoying error can be made invisible when we look at formatting our output in Chapter 29, Sequences of Characters: Strings. For now, however, we'll leave this alone until we have a few more Python skills under our belt.

One consequence is that some fractions are spot-on, while others involve an approximation. Anything that involves halves, quarters, eighths, etc., will be represented precisely. 3.1 has to be approximated, where 3.25 is something that Python handles exactly.

Scientific Notation. Floating point numbers are stored internally using a fraction and an exponent, in a style some textbooks call “scientific notation”. Usual scientific notation uses a power of 10. In the Python language, we write the numbers as if we were using a power of 10. We think of a number like 123000 as 1.23e5. Generally, it means the following, where g is 1.23 and c is 5.

While the Python language allows us to enter our numbers in good-old decimal, our computer doesn't use base 10, it uses base 2. Really, our floating point numbers are converted to the following form.

It's that conversion between the value of g (as entered in base 10) to f (in base 2) back to base 10 that gives us the tiny approximation errors.

One important consequence of this is the need to do some algebra before simply translating a formula into Python. Specifically, if we subtract two nearly-equal floating point numbers, we're going to magnify the importance of the stray error bits that are artifacts of conversion.

Python allows us to use very long integers. Unlike ordinary integers with a limited range, long integers have arbitrary length; they can have as many digits as necessary to represent an exact answer.

We write long integers as a string of digits (no periods) that end in L or l. Upper case L is preferred, since the lower-case l looks too much like the digit 1. Additionally, Python is graceful about converting to long integers when it is necessary.

The trade-off is that long integers will use a lot of memory and the operations are quite slow. For small examples, you'll never notice. But for larger, more complicated programs, you may be unhappy with their slowness.

How many different combinations of 32 bits are there? The answer is there are different combinations of 32 on-off bits. When we try 2**32 in Python, the answer is too large for ordinary integers, and we get an answer in long integers.

>>> 2**32
4294967296L

There are about 4 billion ways to arrange 32 bits. How many bits in 1K of memory? 1024×8 bits. How many combinations of bits are possible in 1K of memory?

2**(1024*8)

I won't attempt to reproduce the output from Python. It has 2,467 digits. There are a lot of different combinations of bits in only 1K of memory. The computer I'm using has 256×1024 K of memory; there are a lot of combinations of bits available in that memory.

We've noted in a couple of places that when you have mixed kinds of numbers Python will coerce the numbers to be all one kind. The rules aren't that complex, but they're important for understanding the semantics of a mathematical formula.

When you mix integers and longs, the integers are coerced to be longs. The idea here is that a long will preserve all the information of an integer, even though the long works more slowly. It's a fair tradeoff. 2+3L is 5L because the 2 was coerced to 2L.

When you mix integers or long integers with floating-point, the integers are coerced to floating-point. Again, the idea is to preserve as much information as possible. However, the floating-point version of a number might not preserve everything.

A floating-point number can represent a vast range of values, but it only has about 17 digits of precision. A long integer can have any number of digits. If your long integer is over 17 digits, some of the precision has to be sacrificed, and it will be the right-most digits of the long integer.

Remember that a floating-point number's right-most digits aren't all perfectly accurate; we're reminded of that every time we see a dangling “0000003”. Consequently, making a floating-point value into a long integer doesn't work out well. Some of the digits on the right-hand end of such a number are more error than precision.

How Coercion Happens. Coercion is something Python does as it evaluates each operator. Here's something you can try to see the effect of these rules.

>>> 2+3/4.0
2.75
>>> 2.0+3/4
2.0

In the first example, the first expression to be evaluated (3/4.0) involves coercing 3 to 3.0, with a result of 0.75. Then the 2 is coerced to 2.0 and the two values added to get 2.75.

In the second example, the first expression to be evaluated (3/4) is done as integer values, with a result of 0. Then this is coerced to 0.0 and added to 2.0 to get 2.0.

As we'll see in the section called “Functions are Factories (really!)” we can force specific conversions if Python's automatic conversions aren't appropriate for our problem.

Besides plain integers, long integers and floating-point numbers, Python also provides for imaginary and complex numbers. These use the European convention of ending with J or j. People who don't use complex numbers should skip this section.

3.14J is an imaginary number = .

A complex number is created by adding a real and an imaginary number: 2 + 14j. Note that Python always prints these in ()'s; for example (2+14j).

The usual rules of complex math work perfectly with these numbers.

>>> (2+3j)*(4+5j)
(-7+22j)

Python even includes the complex conjugate operation on a complex number. Syntactically, the operation conjugate follows the complex number, separated by a dot (.). This makes it a post-fix operator. For example:

>>> 3+2j.conjugate()
(3-2j)

A complex number is a pretty complicated object: it has a real part, an imaginary part, and a number of really unusual operations. Complicated objects force us to use a more sophisticated notation. This is part of a larger, more general syntax pattern that we'll return to this additional syntax in Defining New Objects.

A string is a sequence of characters without a specific meaning. We surround strings with quotes to separate them from surrounding numbers and operators. Unlike a number, which supports arithmetic operations, a string supports different kinds of operations including concatenation and repetition.

You can use either apostrophes (') or quotes (") to surround string values. This gives you plenty of flexibility in what characters are in your strings. You can put an apostrophe into a quoted string, and you can put quotes into an apostrophe'd string. The full set of quoting rules and alternatives, however, will have to wait for Chapter 29, Sequences of Characters: Strings.

Here are some examples of strings. We use apostrophes for the strings that have quotes. We use quotes for the strings that have apostrophes.

"Hello world"
'"The time has come," the walrus said'
"Alice's Adventures in Wonderland"

What if we need both quotes and apostrophes in a single string value? We have to use a technique called an escape. In a quoted string, we may need to escape from the usual meaning of the quote as the end of the string. We use the character \ in front of the quote as an escape. In a quoted string, we use \" to include a quote inside the string. In an apostrophe string, we use \' to embed an apostrophe.

"Larry said, \"Don't do that.\""
'Natalie said, "I won\'t."'

The first example shows a quoted string with a quotation inside it. If we tried "Larry said, "Don't do that."", we would have a syntax error. We'd have a quoted string ("Larry said, "), some random letters (Don't do that.), and another quoted string (""). We have to escape the meaning of the two internal quotes, so we use \" for them.

The second example shows an apostrophe'd string with an apostrophe inside it. To escape the meaning of the apostrophe, we use \'.

String Operators. Strings have two basic operators:

Here are some examples. Note that we had to include spaces in our strings so that the concatenation would look good.

>>> "Walrus" + " and the " + "Carpenter"
'Walrus and the Carpenter'
>>> "Pigs " * 3
'Pigs Pigs Pigs '
>>> "Whether " + ("pigs have " * 2 ) + "wings"
'Whether pigs have pigs have wings'

We'll use strings more heavily in Chapter 13, Seeing Results. It turns out that strings are actually very sophisticated objects, so we'll defer exploring them in depth until Chapter 29, Sequences of Characters: Strings.

For historical and technical reasons, Python supports programming in octal (base 8) and hexadecimal (base 16). I like to think that the early days of computing were dominated by people with 8 or 16 fingers.

You might say to yourself, "Why am I reading this section? I'm not a computer heavyweight!" It turns out that there is a hidden shoal lurking just under the surface of the numbers we've seen so far. The debugging tip, below, is the reason we have to mention this topic.

For much, much more information on bits, bytes, octal and hexadecimal, see Special Ops.

Base 8 - "Octal". A number with a leading 0 (zero) is octal and uses the digits 0 to 7. Here are some examples:

When you enter one of these numbers, Python evaluates it as an expression, and responds in base 10.

>>> 0123
83
>>> 0777
511

An attempt to use digits 8 and 9 in an octal number is illegal and gets you a strange looking error message. In base 8, we only have the digits 0 to 7, the value 8 is 010 (1 in the 8's place, 0 in the 1's place).

>>> 09
SyntaxError: invalid token

In the obscure parlance of language parsing, any symbol, including a number is a token. In this case, the token could not be parsed because it began with a zero, and didn't continue with digits between 0 and 7. It isn't a proper numeric token.

Base 16 - "Hexadecimal". A number with a leading 0x or 0X is hexadecimal, base 16. In order to count in base 16, we'll need 16 distinct digits. Sadly, our alphabet only provides us with ten digits: 0 through 9. The computer folks have solved this by using the letters a-f (or A-F) as the missing 6 digits. This gives us the following way to count in base 16: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, a, b, c, d, e, f, 10, 11, 12, 13, 14, etc. Here are some examples of hexadecimal numbers:

When you enter one of these numbers, Python evaluates it as an expression, and responds in base 10.

>>> 0x53
83
>>> 0x1ff
511
>>> 0xffcc33
16763955

Hex or octal notation can be used for long numbers. 0x234C678D098BAL, for example is 620976988526778L.

Many of the Python processing operations that we might need are provided in the form of functions. Functions are one of the ways that Python lets us specify how to process some data. A function, in a mathematical sense, is a transformation from some input to an output. The mathematicians sometimes call this a mapping, because the function is a kind of map from the input value to the output value.

We looked at the abs function in the previous section. It maps negative and positive numbers to their absolute magnitude, measured as a positive number. The abs function maps -4 to 4, and 3.25 to 3.25.

We'll start out looking at two new mathematical functions, pow and round. Here are some examples of abs, pow and round.

>>> abs(-18)
18
>>> pow(16,3)
4096
>>> round(9.424)
9.0
>>> round(12.57)
13.0

A function is an expression, with the same syntactic role as any other expression, for example 2+3. You can freely combine functions with other expressions to make more complex expressions. Additionally, the arguments to a function can also be expressions. Therefore, we can combine functions into more complex expressions pretty freely. This takes some getting used to, so we'll look at some examples.

>>> 3*abs(-18)
54
>>> pow(8*2,3)*1.5
6144.0
>>> round(66.2/7)
9.0
>>> 8*round(abs(50.25)/4.0,2.0)
100.48

	In the first example, Python has to compute a product. To do this, it must first compute the absolute value of -18. Then it can multiply the absolute value by 3.
	In the second example, Python has to compute a product of a pow function and 1.5. To do this, it must first compute the product of 8 times 2 so that it can raise it to the 3rd power. This is then multiplied by 1.5. You can see that first Python evaluates any expressions that are arguments to the function, then it evaluates the function. Finally, it evaluates the overall expression in which the function occurs.
	In the third example, Python computes the quotient of 66.2 and 7, and then rounds this to the nearest whole number.
	Finally, the fourth example does a whopping calculation that involves several steps. Python has to find the absolute value of 50.25, divide this by 4, round that answer off to two positions and then multiply the result by 8. Whew!

The function names provide a hint as to what they do. Here are the formal definitions, the kind of thing you'll see in the Python reference manuals.

The 〈 and 〉's are how we show that some parts of the syntax are optional. We'll summarize this in Function Syntax Rules.

>>> round( 2.459, 2 )
2.46
>>> round( 2.459 )
2.0

We can use the pow(x, y) function for the same purpose as the ** operator. Here's an example of using pow(x, y) instead of x**y.

>>> 2L**32
4294967296L
>>> pow(2L,32)
4294967296L

Also, pow(x, 0.5) is the square root of x. The pow(x, y) function is one of the built-in functions, while the square root function is only available in the math library. We'll look at the math library in the section called “The math Module — Trig and Logs”.

In the next example we'll get the square root of a number, and then square that value. It'll be a two-step calculation, so we can see each intermediate step.

>>> pow( 2, 0.5 )
1.4142135623730951
>>> _ ** 2
2.0000000000000004

The first question you should have is “what does that _ mean?”

The _ is a Python short-cut. During interactive use, Python uses the name _ to mean the result it just printed. This saves us retyping things over and over. In the case above, the “previous result” was the value of pow( 2, 0.5 ). By definition, we can replace a _ with the entire previous expression to see what is really happening.

>>> pow( 2, 0.5 ) ** 2
2.0000000000000004

Until we start writing scripts, this is a handy thing. When we start writing scripts, we won't be able to use the _, instead we'll use something that's a much more clear and precise.

Let's go back to the previous example: we'll get the square root of a number, and then square that value.

>>> pow( 3, 0.5 )
1.7320508075688772
>>> _ ** 2
2.9999999999999996
>>> pow( 3, 0.5 ) ** 2
2.9999999999999996

Here's a big question: what is that “.9999999999999996” foolishness?

That's the left-overs from the conversion from our decimal notation to the computer's internal binary and back to human-friendly decimal notation. We talked about it briefly in the section called “Floating-Point Numbers, Also Known As Scientific Notation”. If we know the order of magnitude of the result, we can use the round function to clean up this kind of small error. In this case, we know the answer is supposed to be a whole number, so we can round it off.

>>> pow( 3, 0.5 ) ** 2
2.9999999999999996
>>> round( _ )
3.0

>>> round(2.45
...
... )
2.0

Above, we noted that the round function had an optional argument. When something's optional, we can look at it as if there are two forms of the round function: a one-argument version and a two-argument version.

>>> round(678.456)
678.0
>>> round(678.456,2)
678.46
>>> round(678.456,-1)
680

So, rounding off to -1 decimal places means the nearest 10. Rounding off to -2 decimal places is the nearest 100. Pretty handy for doing business reports where we have to round off to the nearest million.

How do we get Python to do specific conversions among our various numeric data types? When we mix whole numbers and floating-point scientific notation, Python automatically converts everything to floating-point. What if we want the floating-point number truncated down to a whole number instead?

Here's another example: what if we want the floating-point number transformed into a long integer instead of the built-in assumption that we want long integers turned into floating-point numbers? How do we control this coercion among numbers?

Python has a number of factory functions that do this kind of number conversion. Each function is a factory that creates a new number from an existing number. Python has numerous factory functions, these are some of the most commonly-used.

These factory functions will also create numbers from string values. When we write programs that read their input from files, we'll see that files mostly have strings. Factory functions will be an important part of reading strings from files and creating numbers from those strings so that we can process them.

float (x ) creates a floating-point number equal to the string or number x. If a string is given, it must be a valid floating-point number: digits, decimal point, and an exponent expression. You can use this function when doing division to prevent getting the simple integer quotient.

For example:

>>> float(22)/7
3.14285714286
>>> 22/7
3
>>> float("6.02E24")
6.0200000000000004e+024

int (x ) creates an integer equal to the string or number x. This will chop off all of the digits to the right of the decimal point in a floating-point number. If a string is given, it must be a valid decimal integer string.

>>> int('1234')
1234
>>> int(3.14159)
3

long (x ) creates a long integer equal to the string or number x. If a string is given, it must be a valid decimal integer. The expression long(2) has the same value as the literal 2L. Examples: long(6.02E23), long(2).

>>> long(2)**64
18446744073709551616L
>>> long(22.0/7.0)
3L

The first example shows the range of values possible with 64-bit integers, available on larger computers. This is a lot more than the paltry two billion available on a 32-bit computer.

Complex Numbers - Math wizards only. Complex is not as simple as the others. A complex number has two parts, real and imaginary. Conversion to complex typically involves two parameters.

The complex function creates a complex number with the real part of real; if the second parameter, imag, is given, this is the imaginary part of the complex number, otherwise the imaginary part is zero. Examples:

>>> complex(3,2)
(3+2j)
>>> complex(4)
(4+0j)

Note that the second parameter, with the imaginary part of the number, is optional. This leads to two different ways to evaluate this function. In the example above, we used both variations.

Conversion from a complex number (effectively two-dimensional) to a one-dimensional integer or float is not directly possible. Typically, you'll use abs(x) to get the absolute value of the complex number. This is the geometric distance from the origin to the point in the complex number plane. The math is straight-forward, but beyond the scope of this introduction to Python.

>>> abs( 3+4j )
5.0

The Formal Definitions. Here are the formal definitions for these factory functions.

If this “complex (real, [imag] )” syntax synopsis with the 〈 and 〉is confusing, you'll need to see Function Syntax Rules.

If the int function turns a string of digits into a proper number, can we do the opposite thing and turn an ordinary number into a string of digits?

The str(x) and repr(x) functions convert any Python object to a string. The str(x) string is typically more readable, where the repr(x) string can help us see what Python is doing under the hood. For most garden-variety numeric values, there is no difference. For the more complex data types, however, the results of repr and str can be different.

Here are some examples of converting floating-point expressions into strings of digits.

>>> str( 22/7.0 )
'3.14285714286'
>>> repr( 355/113. )
'3.1415929203539825'

Here are the formal definitions of these two functions. These aren't very useful now, but we'll return to them time and again as we learn more about how Python works.

The max and min functions accept any number of values and return the largest or smallest of the values. These functions work with any type of data. Be careful when working with strings, because these functions use alphabetical order, which has some surprising consequences.

>>> max( 10, 11, 2 )
11
>>> min( 'asp', 'adder', 'python' )
'adder'
>>> max( '10', '11', '2' )
'2'

The last example (max( '10', '11', '2' )) shows the “alphabetical order of digits” problem. Superficially, this looks like three numbers (10, 11 and 2). But, they are quoted strings, and might as well be words. What would be result of max( 'ba', 'bb', 'c' ) be? Anything surprising about that? The alphabetic order rules apply when we compare string values. If we want the numeric order rules, we have to supply numbers instead of strings.

Here are the formal definitions for these functions.

Python's use of modules is a way to break the solution to a problem down into meaningful chunks. We hinted around about this in Core Coolness. There are dozens of standard Python modules that solve dozens of problems for us. We're not ready to look at modules in any depth, that comes later in Organizing Programs with Modules. This section has just a couple of steps to start using modules so that you can make use of two very simple modules: math and random.

A Python module extends the Python language by adding new classes of objects, new functions and helpful constants. The import statement tells Python to fetch a module, adding that module to the language. For now, we'll use the simplest form: import.

import m

This statement will tell Python to locate the module named m and gather up the definitions. Only the name of the module, m, is added to the local names that we can use. Every name inside module m must be qualified by the module name. We do this by connecting the module name and the function name with a .. When we import module math, we get a cosine function that we refer to with module name dot function name: math.cos.

This module qualification has a cost and a benefit. The cost is that you have to type the module name over and over again. The benefit is that your Python statements are explicit and harbor no assumptions. There are some alternatives to this. We'll cover it when we explore modules in depth.

Another important thing to remember is that you only need to import a module once to tell Python you will be using it. By once, we mean once each time you run the Python program. Each time you exit from the Python program (or turn your computer off, which exits all your programs), everything is forgotten. Next time you run the Python program, you'll need to provide the import statements to add the modules to Python for your current session.

>>> math.cos( math.pi/2 )

Traceback (most recent call last):
  File "<pyshell#2>", line 1, in -toplevel-
    math.cos( math.pi/2 )
NameError: name 'math' is not defined

An Interesting Example. For fun, try this:

import this

The this module is atypical: it doesn't introduce new object classes or function definitions. Instead, well, you see that it does instead of extending Python by adding new definitions.

Even though the this module is atypical, you can still see what happens when you use an extra import. What happens when you try to import this a second time?

The math module defines the common trigonometry and logarithmic functions. It has a few other functions that are handy, like square root. The math module is made available to your programs with:

import math

Since this statement only adds math to the names Python can recognize, you'll need to use the math. prefix to identify the functions which are inside the math module.

Here are a couple of examples of some trigonometry. We're calculating the cosine of 45, 60 and 90 degrees. You can check these on your calculator. Or, if you're my age, you can use a slide rule to confirm that these are correct answers.

>>> import math
>>> math.cos( 45 * math.pi/180 )
0.70710678118654757
>>> math.cos( 60 * math.pi/180 )
0.50000000000000011
>>> math.cos( 90 * math.pi/180 )
6.1230317691118863e-017
>>> round( math.cos( 90*math.pi/180 ), 3 )
0.0

The math module contains the following trigonometric functions. The trig functions all use radians to measure angles. If your problem uses degrees, you'll have to convert from degrees to radians. If your trigonometry is rusty, remember that 2π radians are 360 degrees.

Additionally, the following constants are also provided.

Here's an example of using some of these more advanced math functions. Here is a trig identity for the cosine of 39 degrees. We use 39*math.pi/180 to convert from degrees to radians. We also use the square root function (sqrt).

>>> import math
>>> math.sqrt( 1-math.sin(39*math.pi/180)**2 )
0.77714596145697101
>>> math.cos( 39*math.pi/180 )
0.7771459614569709

Here are some more of these common trigonometric functions, including logarithms, anti-logarithms and square root.

>>> math.exp( math.log(10.0) / 2 )
3.1622776601683795
>>> math.pow( 10.0, 0.5 )
3.1622776601683791
>>> math.sqrt( 10.0 )
3.1622776601683795

The following batch of functions supplement the basic round function with more sophisticated computations on floating-point numbers. You can probably guess from the names what ceiling and floor mean.

>>> math.ceil( 2.1 )
3.0
>>> math.floor( 2.999 )
2.0

The math module contains the following other functions for dealing with floating-point numbers.

Some of the math functions only work for a limited domain of values. Specifically, square root is only defined for non-negative numbers and logarithms are only defined for positive numbers. What does Python do when we violate these rules?

Try the following expressions

math.sqrt(-1)
math.log(-1)
math.log(0)

You'll see one of two kinds of results. The details vary among the operating systems.

Both results amount to the same thing: the result cannot be computed.

The random module defines functions that simulate random events. This includes coin tosses, die rolls and the spins of a Roulette wheel. The random module is made available to your program with:

import random

Since this statement only adds random to the names Python can recognize, you'll need to use the random. prefix on each of the functions in this section.

The randrange is a particularly flexible way to generate a random number in a given range. Here's an example of some of the alternatives. Since the answers are random, your answers may be different from these example answers. This shows a few of many techniques available to generate random data samples in particular ranges.

>>> random.randrange(6)
3
>>> random.randrange(1,7)
3
>>> random.randrange(2,37,2)
6
>>> random.randrange(1,36,2)
5

	We're asking for a random number, n, such that 0 ≤ n < 6. The number will be between 0 and 5, inclusive.
	We're asking for a random number, n, such that 1 ≤ n < 7. The number will be between 1 and 6, inclusive.
	We're asking for a random number, 2n, such that 2 ≤ 2n < 37. The number will be even and between 2 and 36, inclusive.
	We're asking for a random number, 2n+1, such that 1 ≤ 2n+1 < 36. The number will be an odd number between 1 and 35, inclusive.

The random module contains the following functions for working with simple distributions of random numbers. There are several more sophisticated distributions available for more complex kinds of simulations. Casino games only require these functions.

The special operators that we're going to cover in this chapter work on individual bits. First, we'll have to look at what this really means. Then we can look at what the operators do to those things called bits.

A bit is a “binary digit”. The concept of bit closely parallels the concept of decimal digit with one important difference. There are only two binary digits (0 and 1), but there are 10 decimal digits (0 through 9).

Decimal Numbers. Our decimal numbers are a sequence of digits using base 10. Each decimal digit's place value is a power of 10. We have the 1,000's place, the 100's place, the 10's place and the 1's place. A number like 2185 is 2 × 1000 + 1 × 100 + 8 × 10 + 5.

Binary Numbers. Binary numbers are a sequence of binary digits using base 2. Each bit's place value in the number is a power of 2. We have the 256's place, the 128's place, the 64's place, the 32's place, the 16's place, the 8's, 4's, 2's and the 1's place. We can't directly write binary numbers in Python. We'll show them as a series of bits, like this 1-0-0-0-1-0-0-0-1-0-0-1. This starts with a 1 in the 2048's place, a 1 in the 128's place, plus a 1 in the 8's place, plus a 1, which is 2185.

Octal Numbers. Octal numbers use base 8. In Python, we begin octal numbers with a leading zero. Each octal digit's place value is a power of 8. We have the 512's place, the 64's place, the 8's place and the 1's place. A number like 04211 is 4 × 512 + 2 × 64 + 1 × 8 + 1 = 2185 in decimal.

Each group of three bits forms an octal digit. This saves us from writing out all those bits in detail. Instead, we can summarize them.

Binary: 1-0-0  0-1-0  0-0-1  0-0-1
Octal:    4      2      1      1

Hexadecimal Numbers. Hexadecimal numbers use base 16. In Python, we begin hexadecimal numbers with a leading 0x. Since we only have 10 digits, and we need 16 digits, we'll borrow the letters a, b, c, d, e and f to be the extra digits. Each hexadecimal digit's place value is a power of 16. We have the 4096's place, the 256's place, the 16's place and the 1's place. A number like 0x8a9 is 8 × 256 + 10 × 16 + 9 = 2217 in decimal.

Each group of four bits forms a hexadecimal digit. This saves us from writing out all those bits in detail. Instead, we can summarize them.

Binary:      1-0-0-0  1-0-1-0  1-0-0-1
Hexadecimal:    8        a        9

Bytes. A byte is 8 bits. That means that a byte contains bits with place values of 128, 64, 32, 16, 8, 4, 2, 1. If we set all of these bits to 1, we get a value of 255. A byte has 256 distinct values. Computer memory is addressed at the individual byte level, that's why you buy memory in units measured in megabytes or gigabytes.

In addition to small numbers, a single byte can store a single character encoded in ASCII. It takes as many as four bytes to store characters encoded with Unicode.

An integer has 4 bytes, which is 32 bits. In looking at the special operators, we'll look at them using integer values. Python can work with individual bytes, but it does this by unpacking a byte's value and saving it in a full-sized integer.

In Octal and Hexadecimal — Counting by 8's or 16's we saw that Python will accept base 8 or base 16 (octal or hexadecimal) numbers. We begin octal numbers with 0, and use digits 0 though 7. We begin a hexadecimal number with 0x and use digits 0 through 9 and a through f.

Python normally answers us in decimal. How can we ask Python to answer in octal or hexadecimal instead?

The hex(n) function converts its argument to a hexadecimal (base 16) string. A string is used because additional digits are needed beyond 0 through 9; a-f are pressed into service. A leading 0x is placed on the string as a reminder that this is hexadecimal. Here are some examples:

>>> hex(684)
'0x2ac'
>>> hex(1023)
'0x3ff'
>>> 0xffcc33
16763955
>>> hex(_)
'0xffcc33'

Note that the result of the hex function is technically a string, An ordinary number would be presented as a decimal value, and couldn't contain the extra hexadecimal digits. That's why there are apostrophes in our output.

The oct(n) function converts its argument to an octal (base 8) string. A leading 0 is placed on the string as a reminder that this is octal not decimal. Here are some examples:

>>> oct(512)
'01000'
>>> oct(509)
'0775'.

Here are the formal definitions.

More Hexadecimal and Octal tools. The hex and oct functions make a number into a specially-formatted string. The hex function creates a string using the hexadecimal digit characters. The oct uses the octal digits. There is a function which goes the other way: it can convert strings of digit characters into proper numbers so we can do arithmetic.

The int function has two forms. The int (x) form converts a decimal string, x, to an integer. For example int('25') is 25. The int (x, b) form converts a string, x, in base b to an integer.

In case you don't recall how this works, remember that in the number 1985, we're implicitly computing 1*10**3 + 9*10**2 + 8*10 + 5. Each digit has a place value that is a power of some number. That number is the "base" for the numbers we're writing. Python assumes that a string of digits is decimal. A string of digits which begins with 0 is in base 8. A string of digits which begins with 0x is in base 16.

Here are some examples of converting strings that are in other bases to good old base 10 numbers.

>>> int('010101',2)
21
>>> int('321',4)
57
>>> int('2ac',16)
684

In base 2, the place values are 32, 16, 8, 4, 2, 1. The string 10101 is evaluated as 1*16 + 1*4 + 1 = 21.

In base 4, the place values are 16, 4 and 1. The string 321 is evaluated as 3*16 + 2*4 + 1 = 57.

Recall from the section called “Octal and Hexadecimal — Counting by 8's or 16's” that we have to press additional symbols into service to represent base 16 numbers. We use the letters a-f for the digits after 9. The place values are 256, 16, 1; the string 2ac is evaluated as 2*256 + 10*16 + 12 = 684.

While it seems so small, it's really important that numbers in another base are written using strings. To Python, 123 is a decimal number. '123' is a string, and could mean anything. When you say int('123',4), you're telling Python that the string '123' should be interpreted as base 4 number, which maps to 27 in base 10 notation. On the other hand, when you say int('123'), you're telling Python that the string '123' should be interpreted as a base 10 number, which is 123.

We've already seen the usual math operators: +, -, *, /, %, **; as well as a large collection of mathematical functions. While these do a lot, there are still more operators available to us. In this section, we'll look at operators that directly manipulate the binary representation of numbers. The inhabitants of Binome (see Binary Codes are more comfortable with these operators than we are.

We won't wait for the FAQ's to explain why we even have these operators. These operators exist to provide us with a view of the real underlying processor. Consequently, they are used for some rather specialized purposes. We present them because they can help newbies get a better grip on what a computer really is.

In this section, we'll see a lot of hexadecimal and octal numbers. This is because base 16 and base 8 are also nifty short-hand notation for lengthy base 2 numbers. We'll look at hexadecimal and octal numbers first. Then we'll look at the bit-manipulation operators.

There are some other operators available, but, strictly speaking, they're not arithmetic operators, they're logic operations. We'll return to them in Conditional Processing: Only When Necessary.

Precedence. We know one basic precedence rule that applies to multiplication and addition: Python does multiplication first, and addition later. The second rule is that ()'s group things, which can change the precedence rules. 2*3+4 is 10, but 2*(3+4) is 14.

Where do these special operators fit? Are they more important than multiplication? Less important than addition? There isn't a simple rule. Consequently, you'll often need to use ()'s to make sure things work out the way you want.

The ~ operator. The unary ~ operator flops all the bits in a plain or long integer. 1's become 0's and 0's become 1's. Note that this will have unexpected consequences when the bits are interpreted as a decimal number.

>>> ~0x12345678
-305419897
>>> hex(~0x12345678)
'-0x12345679'

What makes this murky is the way Python interprets the number has having a sign. The computer hardware uses a very clever trick to handle signed numbers. First, let's visualize the unsigned, binary number line, it has 4 billion positions. At the left we have all bits set to zero. In the middle we have a value where the 2-billionth place is 1 and all other values are zero. At the right we have all bits set to one.

Figure 11.1. The Basic Number Line

Now, let's redraw the number line with positive and negative signs. Above the line, we put the signed values that Python will show us. Below the line, we put the internal codes used. The positive numbers are what we expected: 0x00000000 is the full 32-bit value for zero, 1 is 0x00000001; no surprise there. Below the 2 billion, we put 0x7fffffff. That's the full 32-bit value for positive 2 billion (try it in Python and see.) Below the -2 billion, we put 0x80000000, the full 32-bit value for -2 billion. Below the -1, we put 0xffffffff.

Figure 11.2. Encoding Signs On The Number Line

This works very nicely. Let's start with -2 (0xfffffffe). We add 1 to this and get -1 (0xffffffff), just what we want. We add 1 to that and get 0x00000000, and we have to carry the 1 into the next place value. However, there is no next place value, the 1 is discarded, and we have a good-old zero.

This technique is called 2's complement. Consequently, the ~ operation is mathematically equivalent to adding 1 and switching the number's sign between positive and negative.

This operator has the same very high precedence as the ordinary negation operation, -. Try the following to see what happens. First, what's the value of -5+4? Now, add the two possible ()'s and see which result is the same: (-5)+4 and -(5+4). The one the produces the same result as -5+4 reveals which way Python performs the operations.

Here are some examples of special ops mixed with ordinary operations.

>>> -5+4
-1
>>> -(5+4)
-9
>>> (-5)+4
-1

The & operator. The binary & operator returns 1-bits everywhere that the two input bits are both 1. Each result bit depends on one input bit and the other input bit both being 1. The following example shows all four combinations of bits that work with the & operator.

>>> 0&0, 1&0, 1&1, 0&1
(0, 0, 1, 0)

Here's the same kind of example, combining sequences of bits. This takes a bit of conversion to base 2 to understand what's going on.

>>> 3 & 5
1

The number 3, in base 2, is 0011. The number 5 is 0101. Let's match up the bits from left to right:

  0 0 1 1
& 0 1 0 1
  -------
  0 0 0 1

This is a very low-priority operator, and almost always needs parentheses when used in an expression with other operators. Here are some examples that show you how & and + combine.

>>> 3&2+3
1
>>> 3&(2+3)
1
>>> (3&2)+3
5

The ^ operator. The binary ^ operator returns a 1-bit if one of the two inputs are 1 but not both. This is sometimes called the exclusive or operation to distinguish it from the inclusive or. Some people write “and/or” to emphasize the inclusive sense of or. They write “either-or” to emphasize the exclusive sense of or.

>>> 3^5
6

Let's look at the individual bits

  0 0 1 1
^ 0 1 0 1
  -------
  0 1 1 0

Which is the binary representation of the number 6.

This is a very low-priority operator, be sure to parenthesize your expression correctly.

The | operator. The binary | operator returns a 1-bit if either of the two inputs is 1. This is sometimes called the inclusive or to distinguish it from the exclusive or.

>>> 3|5
7
>>>

Let's look at the individual bits.

  0 0 1 1
| 0 1 0 1
  -------
  0 1 1 1

Which is the binary representation of the number 7.

This is a very low-priority operator, and almost always needs parentheses when used in an expression with other operators. When we combine &'s and |'s we have to be sure we've grouped them properly. Here's the kind of thing that you'll sometimes see in programs that build up specific patterns of bits.

>>> 3&0x1f|0x80|0x100
387
>>> hex(_)
'0x183'

Let's look at this in a little bit of detail. Our first expression has two or operations, they're the lowest priority operators. The first or operation has 3&0x1f or 0x80. So, Python does the following steps to evaluate this expression.

The << Operator. The << is the left-shift operator. The left argument is the bit pattern to be shifted, the right argument is the number of bits. This is mathematically equivalent to multiplying by a power of two, but much, much faster. Shifting left 3 positions, for example, multiplies the number by 8.

This operator is higher priority than &, ^ and |. Be sure to use parenthesis appropriately.

>>> print 0xA << 2
40

0xA is hexadecimal; the bits are 1-0-1-0. This is 10 in decimal. When we shift this two bits to the left, it's like multiplying by 4. We get bits of 1-0-1-0-0-0. This is 40 in decimal.

The >> Operator. The >> is the right-shift operator. The left argument is the bit pattern to be shifted, the right argument is the number of bits. Python always behaves as though it is running on a 2's complement computer. The left-most bit is always the sign bit, so sign bits are shifted in. This is mathematically equivalent to dividing by a power of two, but much, much faster. Shifting right 4 positions, for example, divides the number by 16.

This operator is higher priority than &, ^ and |. Be sure to use parenthesis appropriately.

>>> print 80 >> 3
10

The number 80, with bits of 1-0-1-0-0-0-0, shifted right 3 bits, yields bits of 1-0-1-0, which is 10 in decimal.

While most features of Python correspond with common expectations from mathematics and other programming languages, the division operator, /, has certain complexities. This is due to the lack of a common expectation for what division should mean. In a mathematics text book, the author will provide additional explanations to clarify the precise meaning of an operator. In a Python program, also, we need to clarify the precise meaning of the / operator.

A basic tenet of Python is that the data determine the result of an operation. For example, when we say 2+3, both numbers are plain integers, and the result is expected to be a plain integer. When we say 2+3.14, Python will coerce the 2 to be the mathematically equivalent 2.0; now both numbers are floating-point, and the answer can be a floating-point number.

This rule meets most of our expectations for ordinary math. However, this doesn't work out well for division because there are two different, conflicting expectations:

There's no best answer and no real compromise. Try the following to see what Python does.

>>> 355/113
>>> 355.0/113
>>> 355/113.0
>>> 355.0/113.0

The Unexpected Integer. Here are two examples of the classical definition of division. We've used the formula for converting 18° Celsius to Fahrenheit. The first version uses integers, and gets an integer result. The second uses floating-point numbers, which means the result is floating-point.

>>> 18*9/5+32
64
>>> 18.0*9.0/5.0 + 32.0
64.4

In this example, we got an inaccurate answer from a formula that we are sure is correct. When a formula has a / operator, the inaccuracy can stem from the use of integers where floating-point numbers were more appropriate. (This can also occur using integers where complex numbers were implicitly expected.)

When we used integers, we got a Fahrenheit temperature of 64, which wasn't precisely correct. When we used floating-point numbers, we got a value of 64.4, which was correct.

The Problem. The problem we have is reconciling the basic rule of Python (data determines the result) and the two conflicting meanings for division. We have a couple of choices for the solution.

We could solve this by using explicit conversions like float(x) or int(x). The idea, however, is for Python be a simple and sparse language, without a dense clutter of conversions to cover the rare case of an unexpected type of data.

Instead, Python has two division operators. For precise fractional results, the / will work nicely.

There are times when we want division to simply compute the quotient. For this reason, Python has a second division operator, //. This more specialized // operator will produces rounded-down integer answers, even if both numbers happen to be floating-point.

Additional Tools. In addition to having two division operators, Python gives us two tools to clarify the precise meaning of the / operator. There are two parts to this: a statement that can be placed in a program, as well as a command-line option that can be used when starting the Python program.

Program Statements to Control /. To ease the transition from older to newer language features, there is a from __future__ import division statement that changes the definition of the / operator from classical (depends on the arguments) to future (always produces floating-point).

Note that __future__ has two underscores before and after future.

>>> 18*9/5+32
64
>>> from __future__ import division
>>> 18*9/5+32
64.4
>>> 18*9//5+32
64

	We used the classical definition of the / operator and got an inaccurate result.
	We set the future definition of the / operator.
	This line shows the new use of the / operator to produce precise floating-point results, even if both arguments are integers.
	This line shows the // operator, which always produces rounded-down results.

The from __future__ statement will set the expectation that your script uses the new-style floating-point division operator. This allows you to start writing programs with version 2.4 that will work correctly with all future versions. By version 3.0, this import statement will no longer be necessary, and will have to be removed from the few modules that used them.

Command Line Options to Control /. Another tool to ease the transition is an option that we can use as part of the python command that starts the Python interpreter. This option can force a particular interpretation of the / operator or warn about incorrect use of the / operator.

The Python command-line option of -Q controls the meaning of the / operator.

Here's how it looks when we start Python with the -Qold option.

MacBook-2:~ slott$ python -Qold
Python 2.4.4 (#1, Oct 18 2006, 10:34:39) 
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 355/113
3
>>> 355./113.
3.1415929203539825
>>> 355.//113.
3.0